Episode 110: Wisdom in Building Data Infrastructure, Lessons From Open-Source Development, The Missing README, and The Future of Data Engineering with Chris Riccomini
Datacast
00:00
Evolution and Developments in Apache Airflow
The chapter explores the speaker's experience with Apache Airflow, covering its early chaotic development, freezing at a certain version, and the improvements introduced in Airflow 2.0. It also discusses comparisons with other orchestration tools, the evolving Python language usage, dynamics in the workflow space, and the importance of tracking data lineage and orchestration for compliance and operational needs.
Play episode from 01:12:41
Transcript
