Super Data Science: ML & AI Podcast with Jon Krohn 825: Data Contracts: The Key to Data Quality, with Chad Sanderson
67 snips
Oct 8, 2024 
Chad Sanderson, CEO of Gable.ai and an expert in data quality and governance, shares insights on the transformative power of data contracts in modern data management. He explains how these contracts clarify expectations for data quality and promote better alignment between data producers and consumers. The conversation dives into 'shifting left' practices that tackle problems early, address concerns about data debt, and the crucial role of human oversight. Chad also highlights storytelling as a tool for data teams to enhance communication and effectiveness.
AI Snips
Chapters
Books
Transcript
Episode notes
Data Contracts for Data Quality
- Data contracts address data quality issues stemming from decoupled data producers and consumers in the cloud era.
- They define expectations like service contracts for APIs, ensuring data reliability and consistency.
Data Quality as Expectation Management
- Data quality isn't about pristine data; it's about managing expectations between producers and consumers.
- Mismatched expectations, like a timestamp format change, cause data quality issues.
Generative AI and Data Quality
- Chad Sanderson's friend's company invested heavily in generative AI but struggled with data quality.
- They couldn't distinguish model hallucinations from incorrect data, highlighting the need for data expectations.
