Scalable and Maintainable Workflows at Lyft with Flyte w/ Haytham AbuelFutuh and Ketan Umare - #343
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
00:00
Managing Data Provenance in Workflows
This chapter explores how the Flyte workflow engine handles data provenance and causal dependencies, emphasizing a unique tracking system for data lineage. It highlights the role of memoization for optimizing computational efficiency and the importance of immutable artifacts for reproducibility in computational tasks.
Transcript
Play full episode
