Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.Nicola's website: https://www.nicolaaskham.com/The Data Governance Podcast: https://www.nicolaaskham.com/podcastNicola's LinkedIn: https://www.linkedin.com/in/nicolaaskham/In this episode, Scott interviewed Nicola Askham, a data governance consultant known simply as The Data Governance Coach and host of The Data Governance Podcast.Some key takeaways/thoughts from Nicola's point of view:The key point of data governance: ensure the data we use is the right data for the right people to better address business challenges. Everything you do with governance should circle back to that.In doing data governance right, you need to set yourself up to take in feedback and iterate. You absolutely won't get everything right upfront. It's crucial to set expectations that your data governance approach will evolve as you learn more.If you see data mesh as being about making better data more accessible to your current data consumers, that's a very big opportunity wasted. Aim to significantly expand your pool of data citizens. Not everyone should be a data scientist but data should play a role in much more people's jobs.To get going in data mesh, you need to get your data governance to "good enough" and start moving forward. Think about what you need - is it the very complicated standard to last the next decade or is it about getting people to understand and trust the data they can now access? Probably the second...To drive buy-in for data governance, you should tailor your message to the audience. It's very hard to have universal appeal around a specific selling point of data governance but data governance can - and should - drive value for everyone.Every data governance approach should be tailored to the organization but it should start from a few building blocks: A) policy; B) processes and standards; and C) roles and responsibilities. (More info below)A centralized data governance team making decisions about what to do with specific data will not scale - they just can't have the context/knowledge needed. So federated governance has been the sensible approach for a long time, it's just not necessarily easy to do right. Or at least it's quite easy to do wrong.Central governance teams are crucial - they make it easy for federated teams to do what's necessary to comply with regulations and internal standards but with as little friction as possible. The central governance team should be a value-add, not a gatekeeper.Make sure teams understand data governance can add significant value to them. Participation is not just some mandate, it has a benefit. Then make sure you are actually providing that value.Data governance has a bad name because of those few using it to put up obstacles. Data governance needs to be about lowering friction, not creating it. Easier said than done of course.In data mesh, you will likely need new roles to handle new data governance needs. Where previously, there were some vague ownership requirements, they should be much more explicit in data mesh.Tying to the point above, you will likely have different sets of requirements under roles across your domains. That's okay. Look to create a standard model for roles and responsibilities and adjust where it needs to be adjusted. It's okay to have non-uniform roles but there needs to be a starting point for domains to go from.When starting out in data mesh, look for a relatively simple first use case but don't only stick to simple use cases early in your journey. It will make it much harder to tackle the difficult use cases later. You don't want a mesh that can't tackle hard but high value use cases!Nicola started the conversation sharing her thoughts on "normal" data governance. What does it even mean? Is that federated, centralized, decentralized, etc.? What she's seen is that functional data governance - at an organization of scale - is not centralized in day-to-day decision making. The central team just can't have the context, the knowledge to make good decisions quickly if at all. But there absolutely needs to be a central team to provide support and knowledge and set federated teams up to succeed. The central team needs to focus on friction-reduction and value-add work. To do that, you need to create standards and processes. But she emphasized keep your frameworks, processes, and standards as simple as possible - no one single, all-encompassing standard please!As an example of functional governance, Nicola talked about how you can't just have universal data quality standards. Every use case may require a different combination of data quality - why optimize for completeness if it's not needed? A central governance team should be focused on helping business stakeholders define aspects of data quality and how to measure it - that way, data consumers can understand the quality of what they consume without learning different standards for each new data source but we aren't setting data quality requirements that aren't helpful or useful.On data mesh specifically, Nicola agrees with most guests: data mesh is very much about the people side. That means the central governance team needs to collaborate with people outside the central team to iterate and improve upon your data governance approach. Feedback leading to improvements is necessary, the governance team can't issue decrees from on high. A part of data mesh that excites her is trying to solve for the age-old challenges of ensuring the data is the right data and that we get it in front of the right people to answer questions about the business - lowering friction to leveraging our data. What "right" means is always somewhat open to interpretation of course.Whether you are doing data mesh or not, Nicola believes data governance can't be about obstacles. That is how data governance got a bad reputation. The phrase should spark joy, not fear or revulsion. Instead, it has to be about making it easy for data consumers to find the right data and then being able to find the right people and documentation to help them understand that data. Governance is about providing low-friction ways to provide access and drive understanding of your data and how to properly use it.One thing Nicola has learned working on a data mesh implementation is that while in data mesh, there are a few new responsibilities that are called out explicitly, that might fall under different roles in different domains. Some responsibilities may fall under a data owner in one domain and under a data steward or mesh data product owner in another. The differing role types she recommends are data owner, mesh data product owner, and data steward. Find a standard setup for roles and responsibilities and then let the domains move responsibilities around as needed - don't make the domains come up with everything from scratch but don't hold on to your standard setup closely either. Everything in data mesh is about iteration and evolution!Scott asked what actually is "good enough" data governance to get moving in a data mesh implementation. Nicola pointed out that no matter what, you won't get your data governance perfect when starting. Especially with something as immature as data mesh is right now. So have clear indications but nothing set in stone. And use her building blocks framework below. Also, think about what capabilities are needed early to drive value: is that some complicated interoperability standards or some data quality definitions/measurement to enable people to understand and trust the data? Probably data quality definitions.According to Nicola, every data governance approach should be tailored to the organization but it should start from a few building blocks: A) policy - as it mandates who will be required to do what and why? Domains just don't do data governance out of the goodness of their hearts; B) processes and standards - lay out what you are trying to achieve and why and then give people an easy way to achieve that. That drives consistency and reduces friction, a win-win; and C) roles and responsibilities - it's very crucial to assign ownership and layout who exactly owns what; we've all been to meetings with no clear next steps and they are almost always a waste of time. Who owns driving things forward? Be clear about it.Some additional data mesh governance advice Nicola gave: 1) Look for a relatively simple first use case. What has a high chance of success where you can also get some momentum and learnings? 2) Don't only look for the simple use cases early in your journey. That can lead to not being able to actually face the hard parts when they come. And with data, of course the hard parts will come. 3) Communicate early and often that you want to collaborate with people and that things will change. Solicit feedback and make constituents part of shaping your governance. 4) Make it clear the central team is there to help and not control - help around compliance, help with reducing friction, etc.A general sentiment that has worked well for Nicola in the past is telling people outside the governance team: if you don't get more value from data governance than you put into, we'll change our data governance frameworks. The governance team may feel the pressure but if you aren't adding value with your governance, outside of regulatory compliance, why are you doing it? Teams will want to participate if you give them a reason to so find the value-add reasons to.Scott asked about a particularly difficult question in data mesh: who owns downstream data products - data products combined from data of upstream data products? For Nicola, if the data isn't transformed, the ownership of the data, even in those downstream data products, should still be whoever owns the originating data product. Essentially, the ownership still flows if the data isn't transformed. But that can cause issues as well if the upstream mesh data product owner doesn't have direct control of how data is used downstream. High communication, trust, and good lineage are necessary.According to Nicola, some people are looking at data mesh as only making data more accessible and usable to existing data consumers. But that's a big missed opportunity. Data mesh can make data more accessible to more employees, driving better decisions. We need data literacy to get to that target outcome but implementing data mesh will lead to a lot of wasted potential if we don't expand the data consumer pool.In wrapping up, Nicola talked about how you can actually drive buy-in for data governance. While trying to sell everyone on upping your data governance game with the same message is not likely to succeed, data governance really does have value for all participants. If it doesn't, as Nicola noted earlier, you need to change your governance approach. So drive that home; tailor the message and speak to - after listening about - their pain points and how you can help them address the pain.Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.Winfried's LinkedIn: https://www.linkedin.com/in/winfried-adalbert-etzel/MetaDAMA podcast (episodes in both English and Norwegian): https://podcasts.apple.com/us/podcast/metadama-en-helhetlig-podcast-om-data-management-i-norden/id1572639573In this episode, Scott interviewed Winfried Etzel, Information Strategy Consultant at Bouvet, Board member of DAMA Norway, and host of the MetaDAMA podcast. Some key takeaways/thoughts from Winfried's point of view:Transformation teams - a team that helps domains to transform and upskill by embedding in the domain for a time - is an exciting pattern for data mesh.Transformation teams can collect information more easily and find patterns as they are closely collaborating with more domains.Think of transformation teams like a personal trainer - they help you get in shape so you learn what exercises to do and how but then you can work on your own. And you can engage them again if you need more help.How can we give guidance and enable change at the same time - the transformation teams shouldn't do the work for the domains but need to work with them. Can we go broad in the organization if we only have a limited number of transformation teams? Do we need to be in that much of a hurry where that's an issue?There's definitely some vendor washing going on around data mesh. It's often difficult to determine what is new and what they've recycled to sound new.In data, we should look to take many learning from software engineering but also other disciplines. History, political science, law, manufacturing, etc. all have something to teach us to better approach sharing our information with each other."Data is a message to people in the future." What do you want to tell them? Data products are how we communicate with those people in the future.Data maturity models are crucial so you can self-reflect - are we really willing and capable to do what we need? And how can we measure how well we are maturing our capabilities?We need to adopt better change management principles in data if we really want to create data citizens. And you should explain how data is important to their role to drive buy-in.Potentially look at using a big bang approach to organizational changes if it is to something like upskilling or even ways of working. Doing a big bang approach for new responsibilities though is probably more challenging and less likely to work.Data ethics as a practice is relatively immature. Companies often don't see a purely business value to training people to act ethically. At most, the return on investment is indirect. Maybe we can figure out how to discuss it relative to retention? How many people want to be creepy and overstep with data simply because they can?Many data mature domains not only want to build everything themselves, they also want to set their own definitions and standards for things like data products. It may be a challenge getting them to participate in data mesh.When finding initial domains to work with in a data mesh implementation, look for domains with the capacity to change - willingness and ability. They will understand what they need to do and why even if they can't do it yet.Winfried shared that in the data mesh meetups for Norway, most of the discussion thus far has been about domain driven design for data, data discoverability, and data as a product. If you are not sure exactly how to approach those topics or what they mean for your organization, you are not alone. And no one is quite sure what federated computational governance means either.In order to do data mesh, Winfried believes people should consider the emerging pattern of Transformation Teams. These are teams that help domains by collaborating to build out their first data product - or possibly more - while also upskilling the team. Scott Hawkins at ITV (episode 48) discussed something similar. The transformation teams also are a great source for finding reuse as they are working with the domains and can more easily recognize patterns. They also can have an excellent perspective on how your implementation is going thus far. We need people that can enable change while giving guidance. A concern though is that the number of transformation teams is limited - how can you go broadly in the organization quickly? Or do you need to be in a rush to do so?In Winfried's view, we need to take far more concepts from software engineering in general in data. Zhamak has stated this multiple times as many, many software engineering practices inspired parts of data mesh. Both mentioned Team Topologies as a crucial tool for change management. But we also should look to other areas outside software too. Winfried sees history as providing a lot of inspiration for data literacy. Political science can be a good way to think about organization design and communication. Law has millennia of examples of good ways to present arguments / one's context. Manufacturing can give us good insight into how we think about products including lifecycle - as Alla Hale discussed in her recent episode.There are many data maturity models, most of which are not that differentiated, according to Winfried. They are still useful when assessing your readiness to do something like data mesh - at the macro and the micro/domain level. Are you organized correctly? Do you have the capability to do what's needed? Do you have the capacity - both meanings: the amount of time and the capability - to change? Are your teams ready and willing to share? Etc.Per Winfried, in general in data, we have not adapted and adopted change management techniques all that well. We need to focus on providing strong learning paths so individuals can change to being "data citizens" as we evolve the overall organization. To inspire people to want to change and get better with data, we need to share with them how data is important to their role. Otherwise, it is homework with no purpose in their heads. Should we go for a big bang approach or incremental, domain by domain? Winfried thinks it shouldn't be only one or the other. When it comes to things like upskilling or even changes to ways of working, big bang might be a better approach so you aren't having to constantly fight against the inertia of the historical patterns and working across so much time to get everyone to a new way of working. But Scott asked about trying to do a big bang approach to new responsibilities and both agreed that can lead to issues.And we need to make sure we keep ethics as something top-of-mind when people learn about data - just because you can doesn't mean you should... Winfried doesn't believe data ethics as a practice has really matured yet. There is of course AI ethics but that is typically about biased inputs, not as much the "should we really even be trying to figure this out?" Companies don't really see the business value in data ethics - which is a challenge that everything has to be about driving business value. There are many cases, especially in the US now, where we need to think about potential harm much more than just potential risk of data exposure. Winfried shared his view on a few things regarding data mesh and domain data maturity. The most data mature domains typically want to do everything themselves - that includes not just building but even things like defining data products. That can lead to challenges when trying to integrate their work into the broader mesh. The domains you should look for are the ones that "understand what they need to do even if they can't do it yet." Those domains have the capacity - the willingness and the ability - to change. Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.Blanca's LinkedIn: https://www.linkedin.com/in/blancamayayo/Pablo's LinkedIn: https://www.linkedin.com/in/pablodoval/In this episode, Scott interviewed Blanca Mayayo - Product Manager, Data Platforms - and Pablo Alvarez Doval - Lead of Data Platforms and Principal Data Architect - at Plain Concepts. From here forward in this write-up, B&P will refer to Blanca and Pablo rather than trying to specifically call out who said which part.Some key takeaways/thoughts from B&P's point of view:It's easy to fall into adding fit-for-purpose capabilities to your data platform but don't. Stay focused on managing your platform as a product - all aspects of it have lifecycles - and you can't try to fit every use case, especially before there is a need.If transformation, especially data transformation, is not tied to the business strategy, that is a major recipe for failure. You likely won't deliver good business outcomes."Beware the proof of concept" - too many try to do a proof without the actual concept. What are you trying to prove and how will you decide/measure if you proved it?You can have everything necessary for a data initiative to succeed lined up - the sponsors, the will, the budget - and still fail. Nothing is 100%.Three common data initiative failure modes: 1) focusing only on the technology aspect and not does it meet needs and can we maintain and pay for it; 2) only treating it as an urgent tactical needs instead of playing into the broader data strategy; and 3) not considering how to actually do change management.Your platform is a product too - data as a product isn't just about mesh data products - think about capability lifecycle and how you communicate upcoming changes - especially deprecation - and help users migrate to the new capabilities.Acclimatize people to change and evolution. Most people in data aren't good with - or at least used to - evolution and preparing for said evolution because the cost of change for data has been so high.To do data as a product, you need the right balance of curious developers, risk/risk management, and capabilities.The most likely places to find reusability in your platform will be the mechanisms around data product production and maintenance - the lineage, CI/CD, data quality monitoring, security/compliance, etc.Reuse is crucial for a data platform - look to have data transformation and storage reuse of course but also really focus on providing templates and then letting users create their own templates. The transformations and handling data don't need to be overly exposed to users.As important background for the conversation, B&P discussed Plain Concepts' journey from a consulting company towards a product company - they had been working with their clients for years, building out a reference architecture for each client's own data platform implementation. They were doing fit-for-purpose, solving very specific challenges with point solutions, not managing the evolution like a product. So the two pillars from data mesh that resonated with them most were data as a product thinking and the self-serve platform - platform thinking is key. They started to push back on their previous practice of putting in additional capabilities to the platform without a key business reason - it's cool tech! - and began to manage the platform much more as a product.Per B&P, part of switching to product and platform thinking was starting to focus on "what to remove" from and what not to add to the platform. Now, they think about where they need to go to support all the users - and align everyone developing the platform on the same vision - instead of trying to support every possible use case. And it's okay for people to use technologies or services that aren't part of the platform if necessary. Shadow IT is only really in the shadows if it's not known about.Another big learning has been the transition of pieces of the platform to keep up with the best available offerings. For example, they started using Hadoop and now mostly use Spark. The platform team was asked to support very specific use cases and did a temporary solution with an eye - from the beginning - on deprecation. And they recommend you start communicating deprecation as early as possible too and work with users to migrate when the time comes. What - typically homegrown - pieces of the platform are not up to the best practices in the industry? One they mentioned specifically was replacing their own data testing and validation system with GreatExpectations.B&P discussed a major failure mode around data transformation in general: your data transformation not being a business strategy transformation. On the micro level, why is the use case you are developing good for the business? Then keep thinking about that up to higher and higher levels of transformation. Look at data mesh - if making a major transformation like data mesh isn't part of the business transformation, will you be able to even make necessary organizational changes? And you must constantly communicate about the transformation - the why and the what - and how the transformation will evolve too. You will learn along the way - leaving no room for evolution clearly doesn't work well.And in general, people in data aren't used to evolution per B&P. In a few of their customers/clients, every other team except the data team has been good with the new ways of change management. And it's understandable, the cost of change in data has been high historically, especially with the data warehouse. To get the data team on your side, thinking evolution is good, you need to collaborate with them and get them to understand the reasons for change. Scott note: I know most listeners, you are the data team that wants to drive change but you are the bold leaders in data :)When talking about change, B&P pointed to a common thread in interviews on this podcast: most data teams have not adopted modern software engineering practices - we're still essentially doing the same thing we were 20-30 years ago. The data architects and data engineers like to play with the bleeding edge technologies but they often aren't trying to adopt them for the right reasons. In B&P's view, instead of bleeding edge, it's better to use more proven tech most of the time unless there really is a capability that's necessary and only available in something emerging. When asked about keeping an eye out for what should be added to the platform regarding reuse, B&P said they were seeing the same general patterns repeatedly, and even there is some parallelism into what other more general software or operational excellence initiatives require. It wasn't that it was exactly the same but once you zoomed out, challenges started to look pretty similar. And more often than not, they were mechanisms around data product production, not the exact data transformation and storage technologies; so lineage, CI/CD, data quality, security, etc.Plain Concepts' data platform is split into three layers of reuse: the first is the technical layer, which they made too technical at first to easily use for people who weren't extremely data engineering capable - don't fall down the same trap. The second layer is all about templates that they've built out through constantly watching for patterns of use. There is even a template catalog. The final layer is all about enabling customers to create their own reuse-focused templates with an SDK. B&P shared a few major underlying issues that will likely cause failure for your data initiatives. First, if your data strategy is only focused on the technical aspect, only on choosing a cool technology that you can't maintain and is too expensive whether that is time, cloud cost, license, etc. Second, trying to react to everything as an urgent tactical need or a short-term change instead of having it play into the broader vision/strategy. And third, not really focusing on change management at all - you need to align incentives, get your early wins to drive momentum, etc. It's a process to drive change."Beware the proof of concept", B&P said - many companies do proof of concepts (PoCs) but they don't have a specific goal or plan. What are you trying to prove out? Why will that drive value? Far too often, organizations cut corners, don't have enough team allocated to the PoC, don't have a strategy or goals, etc. Again, what are you trying to actually prove in your proof of concept? And how will you measure if you proved it? What will you do next if you prove it?When determining if data mesh or anything similar is right for clients, B&P start from the pain points. What are the pain points and then, much more importantly, what is causing those pain points? Much like Scott frequently says: if data team centralization isn't your pain point, don't decentralize your data team! If you are looking at a new approach or technology, what are you really trying to even solve? And even if you line everything up, the sponsors, the budget, the will, things can still go poorly. Nothing is 100%.Other Tidbits:A few additional failure modes mentioned: 1) not continuing investment in your platform - it needs maintenance and innovation. 2) if you aren't having broad-scale collaboration and sharing.Really consider how is IT seen? Is it just about cost optimization or is it a value driver? Can you move to a product mindset?Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereAll music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Sponsored by NextData, Zhamak's company that is helping ease data product creation.For more great content from Zhamak, check out her book on data mesh, a book she collaborated on, her LinkedIn, and her Twitter. Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereData Mesh Radio episode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.Get in touch with Simon and Sunny: data@esynergy.co.ukA webinar (info gated) Sunny and Simon did on data mesh: https://events.esynergy.co.uk/data-mesh-experimentation-to-industrialisation-on-demandSimon's LinkedIn: https://www.linkedin.com/in/simon-massey-82718a3/Sunny's LinkedIn: https://www.linkedin.com/in/sunnysjaisinghani/In this episode, Scott interviewed Sunny Jaisinghani and Simon Massey who are both Principal Consultants at the consulting company esynergy. They have been involved in multiple data mesh implementations including at a large bank. This episode could also have been titled: Aligning Incentives, Reducing Friction, and Continuous Improvement/Value Delivery but it doesn't roll off the tongue very well.From here forward in this write-up, S&S will refer to Simon and Sunny rather than trying to specifically call out who said which part as that leads to confusion.Some key takeaways/thoughts from S&S's points of view:We are all still early in our learnings about how to do data mesh well. There is still a ton left to learn. Which is why people should share what they are learning more broadly. Helping others will help you.Data mesh, whether it's your overall implementation, your platform, your data products, your ways of working, etc. is all about evolution, incremental improvement, iteration, etc. You don't have to get it perfect upfront to drive immense value in the long run. Acclimatize people to iteration and lower the pain of change.Don't go for the big bang approach, find ways to continuously deliver incremental value. That builds the momentum necessary to drive data mesh broad in your organization."Data mesh is 25% technology and 75% ways of working."Once you start to get into a groove with the organizational ways of working, that's when the value force multiplier in doing data mesh starts to take off. Fast time to market with new data products, quick iteration cycles, low-friction cross-domain collaboration, etc. But it takes time to really figure out how to do data mesh in your organization.Your data mesh will inherently have a huge scope. Try to keep that scope as limited as possible as you are getting moving - especially the near term. It is very easy to try to "feature stuff" your data mesh implementation, especially the platform and doubly especially early. Keep complexity out where possible and focus on moving the ball forward, not spending a lot of effort preparing to move the ball forward.If you can't show value from your early data mesh implementation work in 1-2 quarters, you are quite likely to lose any momentum and thus at least a portion of your funding. Find ways to deliver continuous incremental value.For less "data mature" domains, you can use the allure of working with more cutting edge technologies - and upgrading skillsets - to drive buy-in, get software engineers interested in participating.For driving buy-in with moderately data mature teams, ask them how quick iteration on data and reliable data would improve their day-to-day lives. Is there a good use case for your domain you can't handle right now?For the high data maturity/capability domains, they might be reluctant to participate because they like to build things themselves. But data mesh means far less red tape/friction for data consumers, so they will likely pressure the domains to move to data mesh to reduce the friction.In S&S's experience, the most technologically sophisticated domains wanted to do things themselves and were often among the last domains to participate in the data mesh. Don't be shocked if this happens in your organization.Lastly on buy-in, seeing is believing. Many domains will be skeptical. Find those willing and work together to deliver great value. Many of those initial skeptics will now want to participate.Cloud economics and scale are really crucial to achieving value from data mesh. The cost of failure and the scale of failure are much smaller so we can iterate quickly and take more risks. On-prem data mesh is probably not worth the effort in most cases.Similarly, data mesh means pursuing high chance of failure but high reward opportunities around data is much cheaper/easier. Teams don't have to be nearly as worried about an analytical data product being a failure - it wasn't months of time and huge cost, it was a few weeks and not expensive.Think of your mesh data product lifecycle from the start. In alpha mode, you are creating something only for your own domain; it's okay to make drastic changes and if you don't develop it further if there isn't value. Beta is more you bring on one external user and are still early in figuring your specific data product out. Don't push a beta or ESPECIALLY an alpha analytical data product out to the broader mesh.In the same vein, you can treat your data as a product but not arrange it into actual products. Data consumed within the producing domain only doesn't need the same affordances. Don't overcomplicate things.Handoffs between teams, especially where context is crucial, are massive friction points / bottlenecks. Have the producers and consumers directly work together, don't have highly specialized teams like a dedicated ETL team.Use the phrase "analytical data product" or equivalent. If you only say "data product", people often think of other types of data products and aren't as focused on delivering something for your data mesh.If you can really get away with a skunkworks data mesh proof of value, S&S recommend it - Scott is more cautious*. But for most organizations, it will be a top-down mandate, which will make finding willing participants slightly harder.Focus on getting everything out of the way to make technology the easy part. The technology aspect is hard but it's the easiest part of data mesh. No one said data mesh is for the faint of heart or weak of will.Big list of antipatterns below as well.*Note: Scott doesn't recommend a skunkworks approach because there are likely extremely few organizations where you could get far enough to prove value without explicit fundingS&S started off the conversation with a very key point that should be said more often in data mesh: we are all still learning how to do data mesh well and where the sharp edges are, even those who have been doing general distributed data and/or data mesh for multiple years. As Scott said, everyone feels a bit like they are behind the curve but no one is the expert on this yet, it's too early.Trying to lead a data mesh implementation - or even just thinking about data mesh - with a technology-first approach is big mistake many people make per S&S. Data mesh should be close to 75% the ways of working and 25% technology if you are doing it well. If you don't get to a good place with the organizational aspects, those ways of working, your data mesh implementation won't work. Once you start to really find your ways of working, that is when the value force multiplier of mesh really kicks in. But it takes hard work to get there. And, of course, old habits die hard. Change is painful and it will be difficult to change the ways of working.S&S have helped lead multiple data mesh implementations and the initial getting going is always unique to the organization. In their first implementation, they had the budget and freedom to drive innovation enough to find the early adopters, then prove out the value; they were able to go to senior management with proof points when asking for funding to go broader. Most organizations, people won't have that luxury and it will need to be more of a top-down mandate with an executive sponsor. But that might make it harder to find use cases because people see it as a mandate instead of a collaboration at first. And you are likely to lose momentum and at least some funding if you aren't able to show value in 1-2 quarters so get to delivering continuous incremental value early instead of a big bang, back-end loaded approach.On finding the "coalition of the willing", those domains willing to be the early adopter/guinea pigs, S&S recommend looking for people who can understand that having better, more reliable, more high-agility data practices will make their life better. Those tend to be the teams that are semi-mature with data. For the teams that are less mature with data, you can win them over with the allure, especially for those using more legacy technologies, of upskilling and getting to use some cutting edge technologies. On both of those types of teams, S&S have seen people get excited by the rate of iteration possible with data. For the data mature teams, instead of building everything themselves and having lots of checks by governance, legal, etc. to ensure they met every bit of compliance, data mesh means much less red tape - it removes lots of annoying friction for them and they can focus on driving value.S&S recommend in general, in driving buy-in at the domain level, look for valuable use cases that are within the domain itself. That way, you can do much more trial and error because the producer and consumer are one-in-the-same. It also means they will be more incentivized to make it worthy of a mesh data product. And don't push out data products for broad consumption too early - after an alpha stage where it is only consumed within the producing domain, for the beta stage, start with a single use case outside the domain. Once you have the hang of that, think about offering broader access to more consumers serving more use cases but don't push data products out too early.Another incentivization mistake many make - especially the exec sponsors - is focusing too much on the end-state big picture, per S&S. So the CDO or CIO saying "when we have all of our data organized into these beautiful data products with amazing self-serve, think about all the value. Domains, make that happen." But that misses the point of what about now? Why are domains incentivized in the meantime? If all the value is going to be in 2, 3, 5 years, why will they be excited to participate now?So, according to S&S, you need to focus on quickly unlocking business value with each analytical data product as well as with improvements to the platform or other aspects of your implementation. And constantly be iterating towards delivering continuous incremental business value - always be improving what you've got when the work has a good return on investment. Also, believing is seeing. When other domains start to get a positive spotlight from their data-informed results, more domains will want to participate. Back to the data product creation process, S&S emphasized how important "continuous improvement" and iteration to value are in data mesh. You don't have to get it perfect out of the gate. A huge benefit of data mesh is the ability to add - and adjust - incrementally as you learn more to drive more value. Consumers also have to be on board that things may change but it's far better. As stated, the analytical data product creation process S&S recommend is to start with an internal-to-the-domain use case. That way, it doesn't need to be excessively governed or even documented that well. The producer is the owner so they know the needs and the use case. Once you've moved past that alpha stage, don't look to push the data product out wide, find a single use case from another domain and work with them to iterate. Then you can look to publish it to the wider mesh once you really know what you have and what you want to share. And always look to improve and manage your data product like an actual product. "If you don't do the product lifecycle management, it's just a data mart." Data marts didn't go that well when we tried them in the 80s for a lot of those reasons.Incremental improvement and iteration aren't just for your analytical data products according to S&S. Historically, our data setups have been sort of "all or nothing". You either got it right and it created a lot of value or you didn't and it was a flop. The ability to - and the cost of - change in data was very high. Monetary cost, delayed time to insights cost, etc. But with data mesh, it is inherently about iterating, evolving, making improvements, building incrementally, learning and changing, etc.On some anti-patterns, S&S pointed to people trying to do far too much upfront before they start to deliver value. No battle plan survives contact with the enemy. The inherently messy nature of data is our enemy - you will learn far more after you get moving forward than trying to plan and build everything ahead. Set your North Star and start traveling. Focus on CYA - cover your...butt... - governance. Keep out the layers of complexity - no feature stuffing. It's incredibly easy to overcomplicate.Other anti-patterns S&S discussed were: 1) Not ensuring your technology and business sides of every use case are collaborating. Otherwise, you deliver the cool shiny thing that the business side didn't want or need. That's wasted effort. Spend the time to communicate better to prevent that. 2) Going for width of data on the mesh instead of making sure it is ready for consumption. Don't put data on the mesh if there isn't a specific use case for that data. 3) Having a messy data strategy. Becoming "data-driven" isn't a strategy. 4) Moving forward without the organizational maturity to really succeed. 5) As stated earlier, focusing on the technology to the detriment of the organizational aspects. 6) Calling everything an analytical data product. If it's a report or an insight, S&S don't think that should be called a data product. It muddies the water too much. 7) Not adding a clarifier to data product to make it clear exactly what an analytical or mesh data product is. The phrase "data product" is more often misunderstood than understood. 8) Playing on number seven, trying to mix in operational concerns - live transactions - into analytical data products. You will optimize for operational concerns and analytics will become a second class concern. Making analytics a first class concern is pretty crucial to effectively doing data mesh. 9) Chaining too many data products off each other. Downstream of downstream of downstream data products can become a real issue. Try to push use cases upstream where possible.Per S&S, data work has had a high cost - it's been very difficult to get everything right historically and on-prem hardware purchase cycles certainly didn't help. You wanted to make sure something will have a big benefit before looking to invest in it. So there was much less innovation - just going after the low hanging fruit where the chance of success was high. And yet, many projects still failed. However, in data mesh, we can lower the incremental cost to experimentation and new data production very significantly - so you can actually test out more experimental, high chance of failure but high reward use cases. And it means you have the fast time to market capabilities to go after so many analytics opportunities we couldn't in the past.Moving to the cloud means we can fail quicker and cheaper according to S&S. Data mesh is going to be extremely hard and expensive and likely not worth it if you try to do it on-prem in their view. And we iterate from failure to success. Historically, as mentioned, the cost of failure in data initiatives was high - monetary, time, etc. Now, if we fail, we can fail in much smaller ways much more quickly - each analytical data product has a much smaller investment in it than most historical data initiatives. That also means faster feedback, iteration, improvement, etc. So much smaller blast radius and cost of failures means we can be more experimental.S&S referenced one of Zhamak's old figures from early in sharing about data mesh where there are three separate groups, the data producers, the central data team, and the data consumers. The producers have smiley faces, the consumers have a neutral face, and the central data team are all frowning faces. That's because of far too many handoffs between teams - that central team has ownership without context and are a bottleneck/overworked. Jesse Anderson mentioned going outside your team takes 12x as long for work to get done. So there is a big inertia - because it is how many orgs have worked - around trying to have teams specialize in things like ETL or the consumer self-serve platform. But for your own sanity, stay away from handoffs where possible around capabilities.Scott asked about ownership of derived data products and S&S, as referenced earlier, emphasized that many analytical outputs should not be called analytical data products in data mesh. They delved into the concept of treating your data as a product but not necessarily organizing it into actual products. The differentiation was capital p...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf