The Sequel Show cover image

Data Orchestration and Config as Code with Nick Shrock, Creator of Dagster

The Sequel Show

00:00

Constructing Dependency Graphs Using Python

In DAGS to your land you you construct these DAGs these dependency graphs using Python and our config documents are I'm just using a specific to our context. YAML files. Our config documents are used generally for operational context when you want to parameterize the deployed pipelines in some way. So that's a work code right that's also checked in that's also something that I can version. It can be checked in but can also be constructed at Hock if you're like you know kicking off one off jobs from got the web server. That's where our config features really shine because we have like an auto completing yaml editor which is part of the reason why it opens

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app