
Episode 424: Sean Knapp on Dataflow Pipeline Automation
Software Engineering Radio - the podcast for professional software developers
00:00
Is the DAG of All the Steps?
The traditional DAG describes the code that says perform each one of these incremental steps. The sort of new wave is really around declarative data pipelines. And to be declarative, you actually focus less on the steps of like extract from here, do this transform check the states and so on. All of that goes away and is the responsibility of the orchestration system in a declarative model with DAGs.
Transcript
Play full episode