Exploring the Potential of Generative AI and Data Mesh

Please Rate and Review us on your podcast app of choice!

Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Andrew's LinkedIn: https://www.linkedin.com/in/andrewsharp27/

Nicola's LinkedIn: https://www.linkedin.com/in/nicolaaskham/

Kinda's LinkedIn: https://www.linkedin.com/in/kindamaarry/

Jay's LinkedIn: https://www.linkedin.com/in/jaycomoiii/

In this episode, guest host Andrew Sharp, Principal Consultant - Data Governance & Data Protection at The Oakland Group (guest of episode #172) facilitated a discussion with Kinda El Maarry, PhD, Director of Data Governance at Prima (guest of episode #246), Nicola Askham, AKA The Data Governance Coach, an independent data governance consultant (guest of episode #129), and Jay Como, Strategic Advisor at Curate Insights (guest of episode #92). As per usual, all guests were only reflecting their own views.

The topic for this panel was how do we do data governance well including how do we get started around data governance in data mesh. There's a lot to learn about how to improve your governance but there are no blueprints unfortunately. You have to do the work specific to your organization.

Scott note: I wanted to share my takeaways rather than trying to reflect the nuance of the panelists' views individually.

Scott's Top Takeaways:

1. The single biggest misconception around data mesh is that because we call it doing decentralized data, it means decentralize everything. It's even worse when people think it means decentralize everything on day 1. Find your balances as to how far to decentralize different aspects but a central governance function/team will be crucial to doing data mesh right. They need to focus on leverage points and enablement, especially via guardrails, policies, and standards. Each of those three we want to - at least eventually - automate as much as we can. That's the 'computational' part Zhamak mentions.
2. If all we do is empower the domains, there will be chaos. We've seen this with improperly done decentralization in tech repeatedly. It's 'federated' governance and not merely decentralized. We need guiding hands provided by governance to enable the domains to do the right things.
3. In mesh, you have to balance individuality/freedom and consistency. The information encapsulated and how it is encapsulated must be free but why constantly reinvent the wheel where you don't need to? Try to make conformity easy, simple, and the best decision around those places where differentiation isn't a value add.
4. Data mesh creates a great path to _start_ to federate/decentralize data governance. If you try to rush into decentralizing everything upfront, it will lead to chaos. Domains need training wheels around a lot of governance aspects, don't throw them in the deep end.
5. Data ownership is such a crucial aspect to get right in data mesh, especially for governance. That can feel obvious but it's also one of the most common places mesh implementations are suffering or falling down. Be clear what ownership means and help owners understand and take up ownership.
6. Trying to standardize everything instead of create standards that make things easy but are flexible is something that has held back data governance for so long. Data mesh gives us the opportunity to try so many more things out. Take advantage of that and learn along the way.
7. Communication and culture. In data, we are used to thinking that the data communicates itself: look at this perfectly self-describing data model. It's just not the case. And you can't transform your culture and the way your organization communicates overnight. Invest a lot of time and effort into communication and culture if you want to have sustainable value delivery with data mesh.
8. Business people don't want 'data governance', they want to achieve business outcomes. Share with them how the work will help them achieve their goals instead of the specifics of data governance. Speak to outcomes and you will get them excited to do data governance. Focus on the why - the how is exciting and interesting to data folks but most business people don't care!

Other Important Takeaways (many touch on similar points from different aspects):

1. Your mesh governance setup will look different to any other organization's at the very low level and that's to be expected. Your organization is unique - fortunately or unfortunately 😅.
2. It's incredibly easy to get yourself in trouble by trying to be too prescriptive around mesh data governance and following it 'as written'. Zhamak didn't try to write everything as if it was the mesh bible. We're still figuring out how this works. Test what works and be prepared to change what doesn't.
3. Mesh data governance best practices are barely emerging at most. You will need to develop your own theories and practices in some regards. And be prepared for lots of evolution as we as a collective industry figure out how to do this better before we figure out how to do it well.
4. Almost every single organization struggles trying to figure out the mix of responsibilities and roles around data owner, data product owner, data product manager, etc. You will almost certainly not get it perfect to start - test and evolve. Scott note: please, for the love of all that is holy, go talk to say 5 mesh governance leaders and see where they settled on roles and responsibilities so you aren't all repeating the same mistakes.
5. Every domain will have its own quirks. Don't be afraid of changing things, especially roles/responsibilities depending on domain maturity/capabilities. Yes, everything being uniform would be great but it's not realistic.
6. Federate/decentralize at your own pace, don't get ahead of yourself. Some aspects of governance will remain more centralized than you expect as you test what works best for you all along the journey. Again, your equilibriums will be different from other orgs and will change along your journey.
7. Focus on maturity models to understand how to appropriately decentralize aspects of your governance. Not every domain will be as mature either. Again, data mesh is a journey, you don't flick a switch and everything changes.
8. We need mechanisms to at least enable - and potentially enforce - interoperability. If all we have is good data products that don't interoperate, we've only got high quality data silos. Consider how strongly you encourage interoperability and when.
9. Part of a successful data mesh implementation will be about ensuring the data products created are valuable. Highly related, it is important to ask if you are creating the right data products. It's hard to do that ahead of time with governance but governance is key to at least measuring if you are succeeding there.
10. Try to define what good looks like for your domains. It's easier to paint from a reference than from just your mind. It will also encourage more standardization across those things where again freedom and variation don't add value.
11. Nicola said, "You don't go and assign data owners, you identify and engage them." Instead of treating data ownership like a hot potato, you need to figure out who should be the owner and help them step up. That can be different in each domain. Yes, a blueprint that works for every domain would be great but it's not realistic.
12. Many people are excited about what data mesh can bring to their team when they take data ownership. Engage and flame those fires. Help them realize that data ownership can be a great boon, not just more work. Easier said than done of course 😅.
13. On the flip side of course, there are many who will be highly resistant to taking on data ownership. Look to high-level exec support to put the right incentives in place so people will do the right thing. It's also okay to leave some people behind rather than spend all your time trying to convince a brick wall.
14. Data governance in data mesh is harder than traditional data governance. If you are really struggling to do many of the aspects of traditional data governance, really consider if you want to take on doing data governance on hard mode by going the data mesh route. Data mesh isn't for every organization.
15. Conversely, data mesh can be something that forces you to really mature quickly when it comes to data governance. It can be the fire that forges much better practices but that's a risk and it will mean a lot of work if you don't have a great foundation. Scott note: I still go with the previous point that don't try to rush into this if you don’t have a strong foundation but they are interesting points to weigh against each other.
16. An absolutely crucial aspect of good governance is making the implicit explicit. Even make that explicit to your teams; tell them repeatedly to make the implicit explicit. Because that's where you will find friction and misconceptions that will undermine your work.
17. If you are trying to layer in a ton of new roles, it usually means asking for a lot more budget. Think if that will be a hindrance or can you get away with less than perfect ownership to start. It's a risk but if the initial consumers are understanding too that we don't have the budget to nail it perfectly at the start, it can mean you can move ahead with far fewer road blocks.
18. Historically, data work has been often bifurcated with the technical/technology data work in one group's hands and the logic/conceptual/non-technical work in another. Learning to blend the technical and non-technical will probably be harder than you expect. It's a lot about unlearning bad habits and that isn't easy.
19. As often mentioned on this podcast, data literacy in mesh needs to focus on enabling people to actually do the work around data rather than deal specifically with data tools. We need to lower the bar to working with data at the tooling/services/platform and the understanding levels.
20. You may see engineering people get more engaged in governance than they have historically or more than you’d expect. Obviously, invite them into the party to help out. They can add some really great additional expertise and perspective.
21. There is a great misconception around mesh data governance and maturity. Everyone seems to think other organizations are doing governance really well while they are struggling. That's just not true, everyone is struggling. That doesn't mean take it easy but also don't get discouraged.
22. Maintaining mesh momentum is a monumentally material mission. All jokes aside, it is really easy to lose momentum, keep to looking to deliver incremental value and take measured risks - don't swing for the fences at the start.

Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf