Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)Eric's LinkedIn: https://www.linkedin.com/in/ericbroda/Eric's Twitter: @ericmbroda / https://twitter.com/ericmbrodaEric's Medium (multiple posts on data mesh): https://medium.com/@ericbrodaIn this episode, Scott interviews Eric Broda, an Executive Consultant in the financial services space. Eric shared his learnings from aiding a large financial services firm to implement data mesh from the infancy of the project.Eric's big thesis for companies looking to be data-driven is to think of themselves as a platform for connecting supply and demand. The internal company may be the supplier, e.g. if a bank is lending money directly, but more often it is about being the platform - so here to match investors to consumers looking to take out a loan. Lowering the friction between both sides of your constituency is crucial and getting really good at data can help there. To Eric, the technology is like plumbing - you expect it to work but most businesses at the high level don't care about it as long as it works. You don't buy a house for the plumbing.Eric's big point of advice is that you shouldn't underestimate the organizational change required to do something like data mesh right. Plan for the change and don't try to skip the necessary change, that will lead to disaster. Speaking of organizational structure, Eric firmly believes that centralization of data ownership fails Conway's Law. While companies can overcome that with a LOT of effort, most don't get there due to fatigue. When developing a new data product, Eric recommends to first start with expected usage patterns pretty explicitly via a 1:1 relationship model; at least early in a data product's life, the data produced needs to explicitly match the needs of the first target data consumer. This is a departure from data mesh recommended practice but seems to be somewhat of a common emerging pattern, at least in financial services. Eric also stated his belief that master data management - or MDM - "is dead", especially in data mesh. It hasn't ever really worked and it's not worth trying to do it with data mesh. Time will tell on that one.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)Gøran's Twitter: @gorzan / https://twitter.com/gorzanAudun's Twitter: @audunstrand / https://twitter.com/audunstrandGøran's LinkedIn: https://www.linkedin.com/in/g%C3%B8ran-berntsen-66066517/Audun's LinkedIn: https://www.linkedin.com/in/audunstrand/NAIS Platform Website: https://nais.io/In this episode, Scott interviews Audun Fauchald Strand and Gøran Berntsen of NAV. Audun is the Principal Engineer and Gøran is the Product Manager for NAV's NAIS application platform as well as their emerging self-serve data platform for data mesh called NADA.They covered a lot of different topics including: 1) building out the platform; 2) working with consumers to set expectations for common data products; 3) definition of a data product - and how it will evolve; 4) setting the frameworks for producer teams and allowing them to own the production; 5) communicating across teams; and 6) Cake! No really, a secret to success is cake.While NAV is early days in building out their data platform for data mesh, they are taking an interesting approach: work with the developers to set data product expectations and then see how the developers would go about creating those data products. Then, the data platform team will build the platform out to make developer workflows much easier. While Gøran, with a background as a data person, feels the pull to make the self-serve platform as data-centric as possible, he understands the need to make it developer friendly from his time building the application platform with those from a developer background like Audun.They both talked about reducing friction, including via sensible defaults, as a big part of their path forward. Stop trying to make developers come up with everything themselves. While they are still early days on developing those defaults, they are comfortable in their process to get there. And working with developers along the way is key.To start, NAV's definition of a data product is a single table or view. It will probably evolve to be more of a data set focus but they don't see a need to prematurely optimize or overcomplicate. Gøran emphasized the need to have empathy for data producers, to build that into the platform. Teams, whatever the strategic direction, can choose where they focus their time. Don't try to force them to spend it on data, spend the time to really work with them. As Brian McMillan said: find the opportunistic data folks. NAV tried to put analytics or data engineers into the domains but saw them sitting next to the team, not as part of the team. So they decided to rethink. Those data product developers were likely to become overly crucial to serving the data and thus were a likely single point of failure if they moved on.Okay, the most important aspect: Cake! For each team that puts a data product onto the mesh, they give that team cake. As in, an actual cake. It might seem silly but it really does work. It makes it feel less daunting to publish a data product and a bit like you are just having fun. It also means the team can show off a bit when they get their picture out there with their cake in the company Slack. And then people can use that cake picture as a jumping off point for learning more about the data product they just shared. It really is a fun community-building hack.This is a must-listen for anyone involved in building a self-serve platform for the application developers/data product developers.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)Semantic Arts website: https://www.semanticarts.com/Dave's book: https://www.semanticarts.com/software-wasteland/Contact Dave: https://www.semanticarts.com/contact-us/Dave's LinkedIn: https://www.linkedin.com/in/davemccomb/In this episode, Scott interviewed Dave McComb, the President and Co-Founder of Semantic Arts. Scott asked Dave on as part of the continuing deep dive into Domain Driven Design for Data and Data-Centric Application Development as Dave wrote the book on Data-Centric Application Development - literally.Dave's overall argument is that most businesses really have very few "business events", ~500-2000 for even the largest companies. Those large enterprises may have 10K+ applications, each with their own data model and application model, leading to possibly 100M+ data attributes. All that leads to far more complexity than is necessary if companies just focused on building applications from the business events side. They discussed the amount of work an application developer would need to learn to be able to do data-centric application development; while it is mostly about learning data modeling, especially for graph databases, Dave has seen the application developers really not want to move to this model. This has meant a slower roll-out at a number of clients than if they were embracing it.Scott asked about the user experience (UX) in data-centric application development, both for the data producer and data consumer. Per Dave, the UX is pretty lacking, especially on the data producer side so there seems to be a need for better developer tooling for graph databases. Despite the "crude" UX, Dave says he sees data consumers really loving consuming data from a graph. The overall goal of data-centric application development is to provide simplicity and flexibility to organizations as most applications are too rigid for Dave and the system integration is even worse. As mentioned, the first 3 people who fill out a Contact Us on the Semantic Arts website and mention Data Mesh Radio will get a free copy of Dave's book.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Scott shares his emerging data mesh anti-patterns.Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)José's LinkedIn: https://www.linkedin.com/in/jecabeda/José's Twitter: @jecabeda / https://twitter.com/jecabedaIn this episode, Scott interviewed José Cabeda, Data Engineer at Call-center-as-a-service provider Talkdesk. They talked about Talkdesk's start to their data mesh journey and progress so far.When José came across Zhamak's original post, it spoke to a number of the challenges Talkdesk was facing, checking many of the boxes to where they wanted to head. The team started from a single data product and iterated from there. While they are still relatively early in their journey, like every company, they have advanced far past their initial use case.At Talkdesk, a data product is typically a single table or view in Snowflake but the company's North Star is event streaming as their key information storage and sharing mechanism. However, it was sometimes difficult to train people to understand the difference between a business event - something that occurred in the real world - and an event streaming event.José had a few key takeaways and recommendations for those implementing data mesh:1. Change will be constant in a data mesh implementation so it is best to standardize the way people and systems will interact as much as possible. Define expectations!2. Be open to new ideas, there are many challenges ahead so it's best to face them together.3. Use a single universal ID for major concepts like account or business events to make interoperability easier / possible.4. Don't be afraid to slice your data in different ways to serve different use cases.5. To drive buy-in, start with a single use case, whether that is a data product or multiple data products - most people recommend 2-3 data products in your PoC - so you can show why data mesh is a good idea.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)Twitter: @thinh_ha / https://twitter.com/thinh_haLinkedIn: https://www.linkedin.com/in/%E2%98%81%EF%B8%8F-thinh-ha-58945969/Medium post: https://medium.com/google-cloud/10-reasons-why-you-should-not-adopt-data-mesh-7a0b045ea40fIn this episode, Scott interviewed Thinh Ha, Strategic Cloud Engineer at Google Cloud Professional Services. To be clear, Thinh was only representing his own views and was not representing Google/GCP in any way.Scott had asked Thinh to be on after Thinh wrote a post on Medium called 10 Reasons Why You Should Not Adopt Data Mesh (later changed to 10 Reasons Why You Are Not Ready to Adopt Data Mesh). While Thinh is a self-professed "believer" in data mesh, he brings up a number of very reasonable checklist/self-check reasons you wouldn't be ready to move towards data mesh yet.Scott and Thinh go down each of the 10 objections/reasons through the episode and it is advisable to read the article before proceeding. You can see the high level reasons below. There are a lot of very valuable insights into each of the reasons that could make this a 5 page summary so just listen to the episode instead ;)1. You are not operating at a scale where decentralization makes sense2. You do not have a strong business-case for how adopting Data Mesh will deliver business value for individual business units3. You treat Data Mesh as a technical solution with a fixed target rather than an operating model that continuously evolves over time4. Your organizational culture does not empower bottom-up decision-making5. You do not have clearly established roles & responsibilities and incentive structure for distributed data teams6. You do not have a critical mass of data talent7. Your data teams have low engineering maturity8. You expect to find off-the-shelf software to help you adopt Data Mesh9. You do not have buy-in to “shift-left” security, privacy, and compliance10. You do not consider Data Governance to be a core activity to be prioritized against other activities in every data team’s backlogData Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

4 quick things:4 episodes this week; interviews with Thinh Ha (re 10 Reasons You Aren't Ready for Data Mesh article), José Cabeda (Talkdesk's User Journey), and Dave McComb (more on Data-centric Application Design) and one mesh musing on anti-patterns.Still having issues getting transcripts done for episodes. If you are at a vendor that wants to sponsor transcripts, please let me know.Patreon to launch soon, probably Friday.Starting work on the getting started / proof of concept guide very soon.Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated) Azmath's LinkedIn: https://www.linkedin.com/in/azmathpasha/In this episode, Scott interviews Azmath Pasha, member of the Forbes Technology Council, who has 25+ years in implementing large-scale IT projects including at CapGemini and Paradigm Technology.Azmath gave his 3 key measures for data value: cost savings, business value (e.g. driving new initiatives), and data reuse. For data mesh, the long-term value is in the second two but for Azmath, a PoC could be better served focusing on cost savings as it is easier to track and faster to realize.They dove into the concept of data discovery with human interaction, not purely an online experience. Similar to event storming for discovering your domain events (see DDD for Data episodes), discovery as a purely tool-based experience is always likely to be somewhat lacking. Scott was intrigued about this as that aspect of data discovery hasn't been widely discussed.To Azmath, the data product experience, part of what Zhamak calls 'the experience plane', is crucial. It is much harder to drive buy-in if your product is hard to use / has a bad user experience. Azmath's other crucial aspects to getting a data mesh (or any large scale data project) implementation right included: staying tool agnostic so you can remain "future proof"; supporting data producers to reduce time to delivery, especially initial delivery; and looking at your architecture and tool investments over a 5 year time horizon, not just for the short to medium-term.Azmath wrapped up by saying we are entering a new era of using data, we must democratize the data and also look to new metrics for evaluating the business value of data.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Application Strangler pattern (recently renamed Strangler Fig Application pattern): https://martinfowler.com/bliki/StranglerFigApplication.htmlCQRS: https://www.martinfowler.com/bliki/CQRS.htmlKurt's LinkedIn: https://www.linkedin.com/in/kugardiner/nib Group careers page: https://nib.wd3.myworkdayjobs.com/careersIn this episode, Scott interviews Kurt Gardiner, Engineering Manager of Data Engineering at Australian Insurance company nib Group.Kurt shared some insights into nib's journey so far, including the search for something like data mesh before Zhamak published, tool choices (Snowflake, dbt, Fivetran, EventBridge, Kinesis), the slow-role approach to replacing legacy implementation (the "application strangler" pattern mentioned), how they got started, and much more.Much of nib's approach is the small-scale tactical while building incrementally for the bigger strategic focus. E.g. helping teams to design their data products somewhat manually while building the reusable tooling to be far less manual going forward. Along their journey, there was some internal pushback from data consumers, especially those used to consuming from the data warehouse. To do data mesh right, Kurt and Scott both emphasized the need to set things up so they can evolve. That will frustrate or scare some people and it's important to work with them to see why that matters. There also needs to be a high tolerance for failure - you will NOT get everything right on your first go.Kurt also waxed poetic (said nice things) about event streaming patterns, especially CQRS - see link below for more info -, for a useful and scalable pattern that is good for both application development and creating a scalable and useful domain data model. But it requires a complete redesign so it is probably something to slowly introduce where it makes sense, if at all.Some pithy nuggets of wisdom from Kurt that are highly applicable to data mesh:"The single biggest problem in communication is the illusion that it has taken place""Nobody cares what you know until they know that you care"Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf