AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Data Stack Show
158: The Orchestration Layer as the Data Platform Control Plane With Nick Schrock of Dagster Labs
Oct 4, 2023
Nick Schrock, Founder of Dagster Labs, discusses his background in data engineering and the founding of Dagster Labs. They cover topics such as the evolution of data engineering, fragmentation in data infrastructure, the role of orchestration in data platforms, lessons learned from working with GraphQL, different orchestrators in the data infrastructure landscape, the role of MLOps in data engineering, and the future of data teams and orchestration.
01:02:18
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- DAGster Labs aims to revolutionize data infrastructure by providing a declarative and data-oriented approach to solving data engineering challenges.
- The field of data orchestration tools includes a range of options, each with its own strengths and focus, and DAGster Labs sits between airflow and DBT, offering a flexible and accessible solution for data teams.
Deep dives
DAGster Labs: Building an Orchestration Tool for Data Infrastructure
DAGster Labs, founded by Nick Schrock, is an orchestration tool aimed at becoming a control plane for data infrastructure. With a background at Facebook and as one of the co-creators of GraphQL, Nick brings his expertise to the world of data engineering. DAGster Labs provides a declarative and data-oriented approach to solving data engineering challenges. The goal is to make data pipelines more efficient and productive for data and ML engineers, allowing for flexibility in computation and storage. DAGster Labs is focused on bringing value to centralized data teams, with upcoming features including embedded data quality and consumption management. The vision is to create a more integrated and efficient data ecosystem that leverages the power of orchestration in data infrastructure.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode