Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Dan's LinkedIn: https://www.linkedin.com/in/demersdan/Cloud Information Model Standard: https://cloudinformationmodel.org/Cinchy website: https://cinchy.com/In this episode, Scott interviews Dan DeMers, Co-Founder and CEO of Cinchy, a dataware platform / data fabric provider. Dan shares his thoughts on why data-centric application design is the best way to deal with the challenges of applications and analytics needing the same data for different purposes - the current approach is to let the application schema evolve whenever and however necessary and the underlying data applications suffer. Dan's view is "share access to data, not copies."The interview ties loosely with previous interviews re schema/data contracts and data testing as Dan argues using a dataware approach will prevent the issues of an evolving application schema breaking data consumption downstream.It isn't all rosy as this will take a fair bit of work for an organization to move to this approach. Food for thought and the first of a series of interviews re DDD (domain-driven design) for data and data-centric application development.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.In this episode, Scott discusses three concepts that are at best a concern. Consider it a late Grinch-inspired present for Xmas :)Reverse ETL meets a real need for analytical data being pushed into CRM, marketing, and other similar systems. But treating another pipeline as a first order concern is fraught with the same issues of most similar data pipeline treatment: who owns it, how does it evolve, who is observing/monitoring it for uptime and semantic drift, etc.? Should we look to create data products on the mesh to serve those needs instead of another ETL tool?Some organizations implementing data mesh are forcing their domains to consume any analytics from their own data products on the mesh. The good of this is that it aligns the domain with creating a high-quality data products. But will those data products be designed to fit the general organizational needs or specifically the domain's needs?There is an emerging push for software engineers to also own the data modeling. To get to a place where this is even feasible, don't we need far better abstractions for domains to _do_ the data modeling? And will this overload software engineers that are already dealing with a metric buttload of technologies and requirements already? Where would a junior engineer fit in that kind of organization? Does this mean more software engineers on the team -> 2 pizza teams now 3? 4? 5? 10? Maybe we pump the brakes on this for now?Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Jesse's contact info:Email: jesse at tag.bioLinkedIn: https://www.linkedin.com/in/jessepaquette/Twitter: @bzdyelnik / https://twitter.com/bzdyelnikWebsite: https://tag.bio/Tag.bio vendor interview for Data Mesh Learning: https://www.youtube.com/watch?v=acQADu7ttqQIn this episode, Jesse Paquette, Chief Science Officer and Co-founder at Tag.bio - a data platform vendor in the life sciences space, and Scott dive a bit deeper into data quality in general, especially data testing and versioning.You can see the LinkedIn post that sparked this discussion hereJesse recommends a number of things to ensure data quality, especially data testing and versioning. This includes versioning of 1) the code used to create the data (generally the ETL code), 2 the schema, 3) the business logic layer, and 4) timestamping / temporality based versioning.Jesse's general calls to action are 1) make data testing frameworks so testing is much less tedious and time consuming; 2) work with stakeholders to gain trust in the data and then continue the dialogue to keep said trust; and 3) create schema/domain model blueprints so that domains have a starting point - whether they use it is irrelevant but shortening the path to a working domain model is crucial.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Abhi's contact info:Twitter: https://twitter.com/_abhisivasailamLinkedIn: https://www.linkedin.com/in/abhi-sivasailam/Abhi's Data Mesh Learning Meetup: https://www.youtube.com/watch?v=-POiudR2_R0Debezium blog post mentioned by Abhi: https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/In this episode, Abhi Sivasailam, Head of Growth and Analytics at unicorn startup Flexport, and Scott deep dive into data contracts. They covered a LOT of ground including:What is a data contract and how does it relate to API contracts and schema contractsWhy data contracts are so crucial to treating data like a product and keeping data usable by consumersAbhi's rules for a viable data contractThe importance of the analytics engineer to overall data usefulness, especially re domain data ownership in data meshWhy of the socio of the socio-technical approach to data ownership is the most crucial aspectThe lack of proper tooling to monitor and execute data contractsHow to minimize disruptive changes to downstream data consumersThe issues with domains sharing their data as it is persisted/stored in the database instead of sharing the context via the domain modelMuch, much moreData Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Matthew Darwin, a Principal Data Engineer at Slalom Consulting, shares his expertise on platform re-use and data ownership. He emphasizes the potential of leveraging existing technologies when building data platforms. Matthew discusses the journey of implementing data mesh, highlighting that perfection isn’t necessary from the outset. He shares insights on balancing culture and technology, as well as the value of collaboration in data management. His practical advice helps demystify the transition to decentralized data practices.

Olivier Wulveryck, a Senior Consultant at OCTO, dives deep into data schema contracts and their vital role in the data mesh framework. He explains how these contracts ensure data compatibility and enhance trust among users. Olivier emphasizes the importance of versioning and structured communication to maintain data integrity. The discussion also highlights the need for treating data as a product and fostering a culture that prioritizes data quality and governance. Listeners are encouraged to engage with the community for shared learning in this evolving field.

Wannes Rosiers, the CTO of Golazo Group and former leader of DPG Media's data mesh transition, shares his insights on data products. He discusses his framework for categorizing data products and emphasizes the need for a clear purpose in their creation. Wannes highlights the evolution of data products, advocating for ownership and adaptability in data strategies. He also dives into the concept of reverse ETL as a product and the importance of interoperability across domains. Lastly, Wannes encourages community engagement for growth in the data field.

Matthew Darwin, an insightful author with expertise in data platforms and architecture, joins to explore the intricacies of data mesh systems. They discuss the pitfalls of data virtualization for managing data products, emphasizing the need for genuine decentralization rather than superficial changes. The conversation also introduces speculative data products as a clever way to gather consumer feedback, underscoring the importance of consumer-focused documentation in enhancing data engagement and refining offerings.

The hosts are eager to welcome new voices to their discussions on Data Mesh. They emphasize the importance of diverse perspectives in shaping conversations. Listeners are encouraged to provide feedback and suggestions for content that resonates with the community. The call to action invites anyone interested in sharing their insights to join the show. Engage and help create valuable resources for everyone involved!

In this engaging discussion, Paolo Platter, CTO at Agile Lab and a data product development expert, shares insights on migrating existing data products to a data mesh framework. He explores how businesses can identify necessary data products that align with organizational goals. The conversation delves into event storming for data solutions, the importance of tailored implementation plans, and measures for success in data product lifecycle management. Listeners will also discover complex pricing models for data products and how Agile Labs can assist in effective data mesh strategies.