Eric Anderson (@ericmander) interviews Nick Schrock (@schrockn) about Dagster, the open-source data orchestrator for machine learning, analytics, and ETL. Nick is the founder and CEO of Elementl, and is well-known for creating the Project Infrastructure group at Facebook, which spawned GraphQL and React. On today’s episode of Contributor, Nick explains how he set out to fix an inefficiency he identified amongst the complexity of the data infrastructure domain.

In this episode we discuss:

• Dagster’s place in the industry shift towards thinking of data as a software engineering discipline
• Why Nick believes it’s time for the term “data cleaning” to be retired
• The empowerment of Dagster’s instantaneous spin-up process and local development experience
• How a partner integrated Dagster into workflow for ops workers on the warehouse floor
• One user’s testimony that, “what dbt did for our SQL, Dagster did for our Python”

Links:

• Dagster
• Elementl
• GraphQL
• React
• dbt
• Snowflake
• Apache Airflow

People mentioned:

• Lee Byron (@leeb)
• Dan Schafer (@dlschafer)
• Abe Gong (@AbeGong)