Talk Python To Me #454: Data Pipelines with Dagster
4 snips
Mar 21, 2024 Explore data pipelines with Python using Dagster, featuring insights from Pedram Navid. Learn about building efficient data pipelines, orchestrating automation, optimizing deployments with Posit Connect, backfills, partitioning, and popular data tools like DBT, DuckDB, Apache Arrow, and Pollers.
Chapters
Transcript
Episode notes
1 2 3 4 5 6 7 8
Introduction
00:00 • 3min
The Evolution of Data Orchestration and Automation
02:45 • 19min
Building Data Pipelines with Python and a User Interface
21:43 • 7min
Optimizing Data Pipelines and Deployment with Posit Connect
28:29 • 2min
Enhancing Data Pipeline Efficiency with Backfills and Partitioning
30:58 • 12min
Exploring DBT, DuckDB, Apache Arrow, Pollers, and DLT Hub
43:06 • 7min
Exploring the Business Dynamics of the Open Core Model for Successful Cloud Adoption
49:57 • 6min
Concluding Remarks on Getting Started with Dagster and Sponsors Acknowledgment
56:00 • 2min