The Sequel Show cover image

Data Orchestration and Config as Code with Nick Shrock, Creator of Dagster

The Sequel Show

CHAPTER

When Should You Decompose a Python Script?

When do you want to decompose a Python script, say, that's doing data calculations into a sequence of steps or a graph of steps? I think it expresses itself differently at different scales. So in single player mode, the parallelism is a clear win and the need to be able to replay from step two over and over again is something where you get a lot of productivity gains. Once you feel the local productivity hit, when of being able to kind of, like, start from step twoover and over again, that's a good indication that it's time to structure your programming in this fashion.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner