The Data Exchange with Ben Lorica Making Data Engineering Safe for Automation and Agents
12 snips
Nov 13, 2025 Ciro Greco, Co-founder and CEO of Bauplan, discusses revolutionizing data engineering by applying software principles like version control and transactional pipelines to data lakes. He highlights the unique challenges of data work, such as scale and fragmentation, and introduces a git-like branching model for enhanced reproducibility. Ciro emphasizes the importance of transactional guarantees, especially for automated agents, and advocates for a code-first approach to enable safe and efficient interactions with data platforms.
AI Snips
Chapters
Transcript
Episode notes
Apply Software Engineering Rigor To Data
- Treat data engineering like software engineering to get reproducibility, rollbacks, and robustness at scale.
- Software best practices let large teams collaborate without sacrificing application reliability.
Make Data Changes Versioned And Reproducible
- Keep data changes versioned and tied to code so every change is reproducible and auditable.
- You must be able to travel back in time and reproduce deterministic states of the lake.
Enable Team Autonomy With Branches
- Give product and analytics teams their own cloud branches to develop without central-team bottlenecks.
- Monitor and roll back those branches so teams stay independent and safe.