AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Prodity: Product by Design
Lessons in Data Engineering: Scaling, AI, and Open Source with Sandy Ryza
Feb 7, 2025
Sandy Ryza, a lead engineer on Dagster, shares his rich journey from software engineering to data science. He dives into the evolution of data engineering, emphasizing its increasing complexity and the vital role of AI in shaping data platforms. Sandy discusses best practices for managing data, highlighting the integration of software engineering principles. He also reflects on the future of open-source tools and the importance of data ownership in modern infrastructures. His insights offer great value for both seasoned professionals and newcomers in the field.
46:28
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Sandy Ryza emphasizes the necessity of software engineering principles to manage increasing complexity in modern data pipelines and ensure scalable data platforms.
- The rise of unstructured data and AI will reshape data engineering, making intentional platform design and interoperability critical for future success.
Deep dives
Sandy's Journey in Data Engineering
Sandy shares his multifaceted career journey that has revolved around data engineering, starting as a software engineer building tools for complex data sets. After transitioning to a data practitioner role, he faced various challenges that fueled his desire to refocus on creating better tools. This culminated in his current work on Dagster, an orchestration and data management tool aimed at improving data pipelines. His experiences reflect a holistic understanding of both the technical and practical aspects of data engineering, positioning him as a thought leader in the field.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode