AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Importance of Staging and Merging for Data Teams
Git for data, for actual data has been discussed for quite a while now and you mentioned a few fantastic technologies. I'm not convinced that applying versioning to the data itself is actually going to be that much of an impact on the typical data workflow. Another interesting aspect of the staging question is the ability to do that branching and merging workflows where there are some examples of that out in the wild but largely still unsupported.
Data engineering is all about building workflows, pipelines, systems, and interfaces to provide stable and reliable data. Your data can be stable and wrong, but then it isn't reliable. Confidence in your data is achieved through constant validation and testing. Datafold has invested a lot of time into integrating with the workflow of dbt projects to add early verification that the changes you are making are correct. In this episode Gleb Mezhanskiy shares some valuable advice and insights into how you can build reliable and well-tested data assets with dbt and data-diff.
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Special Guest: Gleb Mezhanskiy.
Sponsored By:
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode