Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Max's contact info and related links:LinkedIn: https://www.linkedin.com/in/max-schultze-b11996110/Twitter: @mcs1408 / https://twitter.com/mcs1408Presentations: Similar to content of the book (Data Innovation Summit): https://www.youtube.com/watch?v=rqYFqtztWi4Zalando's Story (Spark AI Summit): https://www.youtube.com/watch?v=UrM8yCjmzzwMax and Arif's 'Data Mesh in Practice' book links:O'Reilly: https://www.oreilly.com/library/view/data-mesh-in/9781098108502/Starburst (free gated download): https://www.starburst.io/info/data-mesh-in-practice-ebook/Max and Arif's upcoming training (Feb 14th): https://www.oreilly.com/live-events/data-mesh-in-practice/0636920508816/0636920068685/Transcript (link) courtesy of Starburst; check out their other data mesh resources herePart of Starburst's Datanova Data Mesh Summit takeover week.Max and Arif's upcoming (Feb 9th) Datanova/Starburst Data Mesh Summit Presentation: https://www.starburst.io/info/datanova2022/In this episode, Scott interviews one of the most prolific content producers in data mesh, Max Schultze, Data Engineering Manager at Fashion E-Tailer Zalando. Max shares a LOT of very valuable advice while reflecting on Zalando's data mesh journey so far two years in. Max recommends starting with empathy building and knowledge sharing but looking for ways to scale versus only 1:1 or small group conversations. He had a clever way to force the hands of data consumers to speak to data producers via an Inverse Conway Maneuver that might work for your org too. Overall, 5 main points/themes emerged: Data mesh is a journeyEmpathy is crucial - have it for those you are working with and work towards building it between teamsTechnology is not the most important aspect of an implementation - no matter how cool it might sound or be to focus on itStart from knowledge sharing, getting people to understand each others' roles and contexts (see empathy!)Try not to get ahead of yourselfHighly recommend giving this one a listen, maybe twice to pick up the nuggets in there.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Very quick one! The Data Mesh Summit (aka Datanova) is taking over Data Mesh Radio for the week! Sign up for an awesome two-day event (Feb 9th and 10th) here.We will have 4 (scheduling willing) amazing guests from the Data Mesh Summit on over the next week: Max Schultze (Zalando), Dr. Colleen Tartow (Starburst), Dr. Daniel Abadi (University of Maryland), and Dr. Teresa Tung (Accenture).In exchange, Starburst is doing a beta, sponsoring transcripts. So please let them know you want more transcripts and again, use the link to sign up to show your support!Again, sign up here.To get Max Schultze + Dr. Arif Wider's 'Data Mesh In Practice' book, click through here

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Mentioned articles and books:Data Management at Scale book: https://www.oreilly.com/library/view/data-management-at/9781492054771/Data Domains — Where do I start? https://towardsdatascience.com/data-domains-where-do-i-start-a6d52fef95d1Implementing Data Mesh on Azure: https://towardsdatascience.com/implementing-data-mesh-on-azure-c01ee94306cdData Domains and Data Products: https://towardsdatascience.com/data-domains-and-data-products-64cc9d28283e"The Blue Book" AKA Domain-Driven Design: Tackling Complexity in the Heart of Software: https://www.oreilly.com/library/view/domain-driven-design-tackling/0321125215/Find Piethein online:LinkedIn: https://www.linkedin.com/in/pietheinstrengholt/Twitter: @phstrengholt / https://twitter.com/phstrengholtScott interviews Piethein Strengholt, Senior Cloud Solution Architect at Microsoft and author of the O'Reilly book Data Management at Scale.Piethein shares his tips and tricks for how to approach Domain Driven Design (DDD) for data. There are puts and takes to each approach so unfortunately for those looking for an easy button, there is a lot to consider. When starting, Piethein recommends looking at your applications and deciding if there is a logical mapping to a single domain or if the application is shared across domains. As you learn more about DDD you can start to approach your domains from multiple other angles to find the best solution for your org.There is a ton of really great advice, too much to sum up well here. Scott highly recommends reading this article on data domains by Piethein before jumping in to the podcast episode. Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Bente's LinkedIn: https://www.linkedin.com/in/bentebusch/Scott interviews Bente Busch, Director of Teams Service Platform - essentially the applications platformm design system, and data platform - at Norwegian government entity NAV (Norwegian Labor and Welfare Department).Bente talked about some really interesting aspects of NAV's data mesh implementation including:The unique challenges of having producers more excited to share the data than consumers demanding more dataChanging the culture from doing "projects" to building productsHow the on-prem enterprise data warehouse just didn't offer them the same agility they needWhere they are in their data mesh journey so farAnd much moreIt's a very interesting look into how data mesh could be applied in government with eventual plans to share information outside of the organization. One very interesting insight is that including those building the application platform in the data mesh self-serve platform build out has been a big win - they are already familiar with application developer workflows and how they think so they are better able to anticipate needs when it comes to building the data platform.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Molly's LinkedIn: https://www.linkedin.com/in/vorwerck/Monte Carlo's blog: https://www.montecarlodata.com/blog/In this episode, Scott interviews Molly Vorwerck, head of content and communications at data observability vendor Monte Carlo. Scott asked Molly to be on as she is a great community member in general and as Monte Carlo is well known in the data space for putting out high quality content on hot-button issues. The idea is to take how Molly and team get their ideas and apply that process to figuring out major pain points internally and creating a cohesive strategy to at least start discussing them.Molly recommends to constantly be interviewing stakeholders. She talks about interviewing people from multiple sides of a challenge, e.g. not just the data consumers but the data producers and the data engineering teams re data challenges. She tries to give people the space to tell their story and asks open-ended questions to truly get their perspective, not arrive at a pre-specified answer.Molly talks about ways to make the other person feel valued by active listening and making the conversation mutually beneficial. It may be a person you want to interview again so building the relationship is crucial, not just extracting info in a one-time manner.Scott and Molly dig a bit into the idea of blameless post mortems and how valuable they can be for doing data mesh, especially to figure out what happened to cause data downtime and how to prevent the same issue in the future.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/In this somewhat controversial episode, part 1 of 2, Scott covers topics re is data mesh right for your organization. This should only be used as a jumping off point for discussions. The 4 segment titles are:Data quality challenges don't necessarily mean it's time for a data meshData Mesh LiteBuilding on a solid foundationHow 'bout now, how 'bout right now? ("Patience you must have, my young Padawan")Take it with a grain of salt!Mentioned content links:Webinar with Zhamak and Sina Jahan - lessons from the trenches with data meshZhamak podcast interview with Barry O'ReillyFlexport Data Mesh Learning meetupPlease Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Chad's contact info:LinkedIn: https://www.linkedin.com/in/chad-sanderson/csanderson.data at gmail.comIn this episode, Scott interviews Chad Sanderson, Head of Product: Data Platform at Convoy. This episode is part of our continuing series on data contracts and related topics. Chad covers a lot of the challenges relative to data quality, both in maintaining quality and in the challenges poor quality data can cause a company that is heavily reliant on data.Chad also shares his tale of trying to implement data mesh at Convoy via a large-scale inverse Conway Maneuver.Chad covered 4 categories of data quality pain, which he calls the "4 Horseman of Data Quality" in this post:Omission - metadata is missing; no tool out today that solves the omission problem, so users have to bounce between too many tools to try to figure out data specifics like where it came from, the specific meaning, what it's trying to convey, etc.Waste: growth of unused, unmaintained, or duplicated data; waste happens when the cost of creating new data is less than using something already createdDivergence: the growing divide between what's going on in "the real world" and what's happening in your data warehouse; your business logic, unless it is constantly maintained and updated, starts to diverge from what is happening to your business so what you show on dashboards and reports no longer matches business realityDowntime: periods of time where the data is missing, wrong, late, etc.; traditionally what most people think of regarding data quality issuesData Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts (most interviews from #32 on) hereProvided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Links from show:Benn's Twitter: @bennstancil / https://twitter.com/bennstancilBenn's Substack: https://benn.substack.com/Benn's post on entities: https://benn.substack.com/p/metadata-money-corporationStarship Technologies Data Mesh Post: https://medium.com/starshiptechnologies/dodging-the-data-bottleneck-data-mesh-at-starship-5925a2de45e6In this episode Scott interviewed Benn Stancil, co-founder and Chief Analytics Officer at Mode about his emerging concept of global definitions that he is calling an "entity" - essentially, how are people defining terms like customer and what does that mean in each instance so people are on the same page. Data mesh also requires some clear collaboration on definitions, whether centralized or decentralized, in the federated governance pillar. This interview was generated from a Twitter conversation here.Benn then went into some details about how he views the importance of having visibility into how data "breaks" so it is much easier to identify and fix and that limiting custom integration is crucial so when you fix at the source, it properly propagates downstream. They wrapped up discussing the need to make data producers' lives easier while simultaneously doing the same with data consumers.Overall, there are some agreements and disagreements and Scott came out thinking more about what are the real causes of pain that would make a full journey to data mesh make sense. It's a good episode to see some of the challenges people are trying to tackle outside of the data mesh community.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him at community at datameshlearning.com or on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts (most interviews from #32 on) hereProvided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Juan's contact info and related links:Email: juan at data.worldTwitter: @juansequeda / https://twitter.com/juansequedaLinkedIn: https://www.linkedin.com/in/juansequeda/Catalog & Cocktails Podcast: https://data.world/podcasts/Juan's post about Zhamak's appearance on the Data Engineering Podcast: https://www.linkedin.com/pulse/my-takeaways-data-engineering-podcast-episode-mesh-zhamak-sequeda/Juan's post about knowledge first: https://www.linkedin.com/feed/update/urn:li:activity:6884179569277059072/Standards related links:Dublin Core Metadata Initiative: https://dublincore.org/RDF (Resoruce Description Framework): https://www.w3.org/2001/sw/wiki/RDFOWL (Web Ontology Language): https://www.w3.org/OWL/PROV-O: The PROV Ontology: https://www.w3.org/TR/prov-o/In this episode, Scott interviews Juan Sequeda, Principal Scientist at data.world and co-host of the Catalog and Cocktails podcast. They discussed Juan's knowledge first approach: putting the meaning and value of the data first instead of focusing on the amount of data we are handling/producing. Knowledge first has 3 components, 1) context, 2) people, and 3) relationships. Juan is a big proponent of knowledge graphs and the relationships side is one many people miss.Juan also gave some thoughts on what his approach to data mesh hinges on: treating data as a product and finding a balance between centralization and decentralization for all the aspects of building out an implementation. Juan mentioned Intuit's approach of fixed, flexible/extensible, or customizable as a good general tool and to look for (and embrace) what he calls intellectual friction.Lastly, Juan and Scott talked about the general drive to reduce toil, of reinventing the wheel re data interoperability and standard schemas in data mesh. Juan points to a lot of existing research and standards - e.g. RDF, OWL, and many more (see below) - as a starting point.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him at community at datameshlearning.com or on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Peter's contact info and relevant links:Email: peter at cloudshuttle.com.auLinkedIn: https://www.linkedin.com/in/peterhanssens/Twitter: @petehanssens / https://twitter.com/petehanssensCloud Shuttle website: https://www.cloudshuttle.com.au/Sydney Data Engineering Community: https://sydneydataengineers.github.io/DataEngBytes Conference: https://dataengconf.com.au/Article mentioned by Francois Nguyen (CIO of L'Oreal): https://francois-nguyen.blog/2021/03/07/towards-a-data-mesh-part-1-data-domains-and-teams-topologies/In this episode, Scott interviews Peter Hanssens, Founder and Solutions Architect at cloud/serverless consulting company Cloud Shuttle. Peter also runs a few large data engineering focused communities (meetups, Slack, conferences) in Australia. Peter had reached out about how can startups and SMEs (small to medium enterprises) get the benefits of data mesh without the costs of building a solution fit for a 10,000+ employee company. Peter does a great job seeking information on behalf of his constituents :)We covered a number of topics including: The needs for cultural change as technology will only get you so farThe beauty of pay-per-use solutions for startups/SMEs - and where there are still gaps in the marketThe challenges of not having a large pool of data engineers to build and manage a platform - or to help domains model their dataThe benefits of centralization until it starts to cause bottlenecksThe importance of tracking lineage for upstream producers/domains - to see who is using your data and why/howAnd much, much moreI think you will really enjoy Peter's perspectives and there are some useful conclusions if not a perfect blueprint for startups.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf