CaSE: Conversations about Software Engineering Data Marketplace, Data Products and Data Contracts
4 snips
Sep 22, 2025 Simon Harra, CEO of Entropy Data and co-creator of the data mesh manager, dives into the transformative world of data management. He discusses the evolution from software engineering to decentralized data thinking, emphasizing the importance of data contracts. Simon breaks down the anatomy of these contracts, covering quality guarantees and validation processes. He then shares insights on building a centralized data marketplace and explores future possibilities like AI-enhanced discovery and governance automation. A fascinating look into the future of data products!
AI Snips
Chapters
Transcript
Episode notes
From Java To Data By Experiment
- Simon transitioned from Java backend work to data after his backlog went dry and his team started experimenting with BigQuery.
- That experiment turned them into a hypothesis-driven product team owning both operational and analytical data end-to-end.
Marketplace As The Coordination Hub
- A data marketplace is the central coordination point for federated data teams, governance and discovery.
- The marketplace should surface data products and expose guarantees captured by data contracts to build trust.
Require Conscious Publication
- Do not auto-generate data products from discovered tables; require owners to consciously publish curated products.
- Make publishing easy with tooling, but keep the act deliberate to ensure quality and ownership.