Guest Sandy Ryza, an expert in machine learning pipelines, discusses the role of orchestrators in the lifecycle of data, changes in data ops and MLOps, data cleaning, and the overview of Dagster. They also explore the difference between data assets and tasks in data pipelines, defining lineage and data assets in Dagster, and the benefits of a unified orchestration framework. Additionally, they touch on orchestration in the development phase and the emergence of the analytics engineer role.

Sandy Ryza, a lead engineer on Dagster, shares his rich journey from software engineering to data science. He dives into the evolution of data engineering, emphasizing its increasing complexity and the vital role of AI in shaping data platforms. Sandy discusses best practices for managing data, highlighting the integration of software engineering principles. He also reflects on the future of open-source tools and the importance of data ownership in modern infrastructures. His insights offer great value for both seasoned professionals and newcomers in the field.

Sandy Ryza from Dagster Labs discusses the role of an orchestrator in Data Ops and ML Operations. They also emphasize the need for diverse solutions in the ML operations space.