Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.PredictiveUX website: https://www.predictiveux.com/PredictiveUX partnered meetup: https://www.meetup.com/hexagon-ux-dc-chapter/Karen LinkedIn: https://www.linkedin.com/in/karenpassmore/Karen Email: karen at predictiveux.comSteve LinkedIn: https://www.linkedin.com/in/stephenstesney/Steve Email: sstesney at predictiveux.comIn this episode, Scott interviewed Karen Passmore (CEO) and Steve Stesney (Data Product Lead) at consulting firm PredictiveUX. They touched on a lot of different topics - a key theme throughout is the importance of the user experience in data mesh, for both data product producers and consumers.Karen highlighted some parallels between data mesh and content management projects and how to take some key learnings from the past and apply them to data mesh implementations. They discussed the importance of providing your internal people with the right content at their current point in their learning journey - a successful implementation of data mesh requires making it far easier and more scalable to share incremental work artifacts and knowledge - it also means your crucial company knowledge actually gets documented properly.Steve talked about some historic challenges he had personally with decentralized teams - if you don't manage the cross domain collaboration, both at the business and the technical implementation levels, it is a major pain to stich your data together from all those sources. So there needs to be good alignment on interoperability. Basically, data mesh without a good interoperability strategy is just high quality data silos.Karen and Steve both emphasized the importance of UX (user experience) for driving adoption. You can have the best solution in the world but if the users don't want it, it's not going to be successful. So working with them throughout the process is crucial to get to a successful implementation, whether that is data mesh or not.Karen wrapped up by emphasizing the need to be patient and to not expect the same results or try to copy the exact path of other organizations implementing a data mesh. Every organization is very unique and you need to figure out what might work for your organization. Take learnings, not exact blueprints.The last key point to extract is the need for multiple communication methods, especially for data requests. There may be some overlap but it's a great way to ensure reliability and scalability of your business processes.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Andrew's Medium post called Improving Data Quality with Data Contracts: https://medium.com/gocardless-tech/improving-data-quality-with-data-contracts-238041e35698LinkedIn: https://www.linkedin.com/in/andrewrhysjones/Twitter: @andrewrjones / https://twitter.com/andrewrjonesIn this episode, Scott interviews Andrew Jones, Tech Lead of the Data Infrastructure Team at GoCardless. Andrew shares the story of how operational system changes kept breaking downstream data consumption (sound familiar?), especially using CDC. The software engineers couldn't easily use the CDC tooling and the data engineers could easily use the data from it either as CDC didn't structure the data for easy consumption. Andrew wasn't really sure how other people were handling taking the API contract concept and leveraging it for data but started building out some generic simple tooling to let consumers and producers feel somewhat comfortable with their data contracts. A big revelation was in helping data consumers make better asks for data. The data consumers weren't used to asking the producers for data, especially in a reliable and scalable way (sound familiar?). GoCardless now has an actual standard form for data consumers to use to request data and that is working quite well.GoCardless plans to completely remove their CDC architecture by 3Q of this year to replace with data contracts. They are focusing on providing tooling to give domains the autonomy to serve data to consumers in their own way. While it isn't data mesh, especially with the lack of interoperability between data products and lack of source/producer-aligned data products, it seems to be working for GoCardless thus far.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Danilo's Webinar with Zhamak called "Data mesh and domain ownership": https://www.thoughtworks.com/en-us/about-us/events/webinars/core-principles-of-data-mesh/data-mesh-and-domain-ownershipVladik Khononov, 7 Years of DDD: https://www.youtube.com/watch?v=h_HjtYAH0AIDanilo LinkedIn: https://www.linkedin.com/in/danilosato/Danilo Twitter: @dtsato / https://twitter.com/dtsatoAndrew LinkedIn: https://www.linkedin.com/in/andrewharmellaw/Andrew Twitter: @al94781 / https://twitter.com/al94781In this episode, Scott interviews two Domain Driven Design (DDD) experts from Thoughtworks - Danilo Sato Director and the Head of Data & AI (part of Office of the CTO) and Andrew Harmel-Law, Technical Principal. This was a further delve into Domain Driven Design for Data after the conversations with Paolo Platter and Piethein Strengholt.Danilo and Andrew gave a lot of great information about Event Storming, domain definitions and boundaries, ubiquitous language, and so much more but the main theme was "just get people to talk to each other".DDD is about bridging the gap between how the tech people talk and how the business/business people talk; if you are doing it right, both sides can understand each other and then the engineers can implement those business process learnings as part of the code.For an initial PoC, Danilo recommends starting with 2-3 data products. It is better if you can do the PoC across multiple domains but it isn't necessary. Validate value and do it quickly. As Andrew mentions, the earlier you can show value, the less pressure there is overall. Look for the initial quick wins while also building for the long-term.One key thing to remember, per Danilo, when doing DDD for data and data mesh in general: it is always an iterative process. Andrew briefly discussed a way to do DDD in more of a guerilla style than the blue/red books (well known DDD guides). Don't get ahead of yourself as Max Schultze mentioned in his episode. Do not let the size of the eventual task throw you into analysis paralysis. Andrew talked a lot about how normalization and strong abstractions on the application side make it very difficult to re-add the context lost when you normalize. Both Andrew and Danilo talked about the need to embrace complexity. If you want context, you have to accept there will be complexity. In the pursuit of simplification, you lose the richness, and that is VERY hard to reconstruct afterwards.Some practical advice for boundary definition is that the boundaries need to be very clear but malleable. Build everything with an eye that it will evolve. Before you start splitting into many 2 pizza teams, look at the big picture and select some coarse-grained boundaries. It is MUCH easier to split later than it is to glue things back together.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Angelo's LinkedIn: https://www.linkedin.com/in/angelomartelli/In this episode, Scott interviews Angelo Martelli, Group Leader of Data Services at logistics company Vanderlande. Angelo laid out his framework for driving data mesh buy-in internally at Vanderlande which helped them take the idea from a small group to a company-wide initiative:Start with proving there is a problem that you are trying to solve - if everything is functioning well, why focus your efforts on that area instead of trying to fix another? Your proof should be as fact-based as possible, e.g. how long does it take to make a change to your data warehouse. Focus on proving that your incremental investments are driving sub-linear returns. Other areas to look to prove problems: how many people are involved in a change to your data warehouse, percent of time spent on regression testing versus development, mean time to resolution of challenges, etc. Once you have some proof, you need to work towards understanding the problem you are trying to solve. It's not "deploying a data mesh", it's scaling the organization to be agile relative to data and be able to make more (and better) data-informed decisions. Next, you need to understand your organization. Who are the right people that can help you? How does your organization work relative to culture and process? Which domains are struggling and how? Tie the implementation goals to the actual business challenges.Then, you need to demystify data mesh, make it easy to understand for people not well versed in data - what are we actually trying to accomplish and why? Last, make it concrete / prove it out. Make a few data products, make a simple platform for folks to use. Angelo then recommends that once you have momentum, sharing a very clear vision is crucial. Not just sharing in a document but actually having conversations to really make sure the context and vision is understood. Data mesh is about collaboration, you must work together so it is imperative to make expectations very clear. Similar to Abhi Sivasailam, Angelo also stressed the importance of the domain data model and abstracting that away from the application model(s). The business model is what matters for data. All of that and so much more. Also, Angelo gives a shout out to the usefulness of the Data Mesh Learning community. 😎Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Brian McMillan, a former Enterprise Architect, discusses his book 'Building Data Products: Introduction to Data and Analytics Engineering for Non-Programmers' and shares insights on transitioning from data manipulation to shareable orchestration, collaborating on data products, improving SQL skills, preventing shadow IT, and approaching data culture in a different way.

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Somewhat short episode. Scott emphasizes that you aren't the only one facing challenges with data mesh. You aren't behind the curve, you are ahead of it - data mesh is bleeding edge. But with a bleeding edge, there is some blood...Also, you need to think about how others' implementations works for them. Trying to copy-paste another organization's implementation is going to lead to a failed implementation in yours. Take the learnings away and think about how to apply them to your organization.Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Sheetal's (and Divya and Madhu from Thoughtworks) Data Mesh Learning Meetup presentation: https://www.youtube.com/watch?v=5btUaLPdaNkSheetal's post re data mesh and using DataHub: https://blog.datahubproject.io/enabling-data-discovery-in-a-data-mesh-the-saxo-journey-451b06969c8fSheetal's LinkedIn: https://www.linkedin.com/in/sheetalpratik/Data Governance using Data Mesh paper: https://easychair.org/publications/preprint/qZ3mIn this episode, Scott interviews Sheetal Pratik, Director of Engineering, Data Integration at Adidas. If Sheetal's name sounds familiar to data mesh officianados, she presented at the Data Mesh Learning meetup in August 2021.Sheetal is passionate about giving companies the permission AND a workable plan for getting started with data mesh. She covered a wide range of things regarding getting starting but a few really stood out:Don't try to tackle tomorrow's challenges todayBreak your implementation into phases: development, adoption, and scalingStart your initial data mesh MVP with a simple data product with a simple schema - your goal is to develop the "muscles" around creating and deploying data products rather than shooting for a high-value product firstKeep to a reasonable budget and prove viability and valueSheetal also covered how much you really have to have in place to create and evaluate your MVP. There will always be evolution and change and your organization has to be ready for that. That can be frightening or inspirational. Sheetal chooses it as inspirational - it gives you the freedom to move quickly as long as the organization understands that things will change in the future. She wrapped up with saying that data mesh shouldn't be scary, you should be excited about this journey and what it can mean for your organization. Get an MVP out the door, it will take time, don't get ahead of yourself. Data mesh success can happen if you let it. Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Dr. Abadi's blog post on data federalization and data virtualization: https://blog.starburst.io/data-federation-and-data-virtualization-never-worked-in-the-past-but-now-its-differentDr. Abadi's contact info:LinkedIn: Twitter: @daniel_abadi / https://twitter.com/daniel_abadiStarburst blog posts: https://blog.starburst.io/author/daniel-abadiDon't forget to catch Dr. Abadi at Datanova - the Data Mesh Summit on Feb 9-10th. Thanks to Starburst for sponsoring the transcripts for Data Mesh Radio, check out the transcript here. And check out Starburst's other free data mesh resources here.In this episode, Scott interviewed Dr. Daniel Abadi, the Darnell-Kanal Computer Science Professor at the University of Maryland with a focus on scalable data management research. Dr. Abadi will be presenting next week at the Data Mesh Summit on Data Fabric and Data Mesh alongside Zhamak and Sanjeev Mohan.This was a pretty wide ranging and free wheeling conversation about data virtualization in general and how it can be used in data mesh. Both agreed that there are many places where data virtualization can play in data mesh, whether in extracting information from operational systems, stitching together a data product once data processing has been done, or at the mesh experience plane re combining data across multiple data products. Dr. Abadi specifically mentions something like a query fabric that makes use of a data virtualization approach, not just tools that only do data virtualization.There is a natural side effect of having multiple different technologies in use - when you give the domains the ability to use what they choose, the difficulty of combining data from multiple sources needs to be solved. There is always a balance between how much you just copy data and how much you can access in the source system and data virtualization can give a few more options rather than all or nothing.As data virtualization has been around as a concept for 30+ years, there is a lot of baggage with the term but Dr. Abadi sees there being recent advancements that mean more people should take a second look at where they can be useful. But warns to do your homework and really think through whether they fit your use case. A query fabric can make your user experience much more pleasant. Trying to create data products entirely within a data virtualization platform probably won't be, at least according to Scott.Additional topics included retransmitting or reprocessing data, versioning, the importance of denormalizing data for analytics and how that plays with data virtualization, and much more. It is a really fascinating deep dive into the history of computing and how it impacts what we are trying to do today.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Colleen's contact info:Email: colleen at starburst.ioLinkedIn: https://www.linkedin.com/in/colleen-tartow-phd/Twitter: @CTartow / https://twitter.com/CTartowColleen's blog: https://thesequel.substack.com/The Data Buffet blog post: https://thesequel.substack.com/p/the-endless-data-buffetDon't forget to catch Colleen at Datanova - the Data Mesh Summit on Feb 9-10th. Thanks to Starburst for sponsoring the transcripts for Data Mesh Radio, check out the transcript here. And check out Starburst's other free data mesh resources here.In this episode, Scott interviews Dr. Colleen Tartow, Director of Engineering at Starburst. They chatted about the fun and usefulness of food-related analogies to data mesh - you want it to work like a brunch buffet as that is basically a perfect Saturday in Colleen's eyes. And Scott shared the concept of a grocery store - intentional food preparation with different degrees of ingredient and meal preparedness. Colleen and Scott then covered the "Modern Data Stack" and its relevance to data mesh - in Colleen's eyes, the Modern Data Stack isn't all that modern, it is just the same old paradigm of data teams trying to do their best to work with the output of application/operational data stores and systems that aren't designed with the data in mind. Scott somewhat agreed with his own spin.Colleen shared her 4 S paradigm re doing data well: speed, simplicity, scalability, and SQL. Yes, slightly Starburst self-serving (4 more S words!) but still relevant and interesting. They wrapped up with trying to figure out how much of data mesh can companies get away with not doing, which seems to be the topic on everyone's mind. More questions than answers there...Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf