GOTO - The Brightest Minds in Tech Incremental Design, DevOps, Microservices & CICD • Michael Nygard & Dave Farley
Oct 3, 2025
Michael Nygard, General Manager of Data at Nubank and author of Release It!, teams up with Dave Farley, a pioneer of DevOps and founder of Continuous Delivery Ltd. They dive into how data challenges are often overlooked in DevOps, the modularity of data products compared to APIs, and the intricacies of automatic ETL strategies. They discuss the need for stable abstract interfaces in platform engineering and the balance between universal design and practical limitations. Their insights highlight the evolution of software practices and the complexities of modern software development.
AI Snips
Chapters
Books
Transcript
Episode notes
Incremental Changes For Big Data Estates
- Large data estates force incremental, parallel work with clear boundaries and tooling.
- You must preserve critical operations (eg. financial reporting) while changing systems incrementally.
Treat Data Products As APIs
- Modularity is essential, but data semantics lack good abstraction tools today.
- Treat data products like APIs to create stable interfaces teams can rely on.
Automatic ETL From Microservice Schemas
- At Nubank every microservice declares its schema and ETL is generated automatically on deployment.
- They extract to Parquet in S3 and are moving toward Delta Lake and incremental change-data-capture.
