AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is the DAG of All the Steps?
The traditional DAG describes the code that says perform each one of these incremental steps. The sort of new wave is really around declarative data pipelines. And to be declarative, you actually focus less on the steps of like extract from here, do this transform check the states and so on. All of that goes away and is the responsibility of the orchestration system in a declarative model with DAGs.