The Importance of Blending in the Agile Approach for DataMesh

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Eric's LinkedIn: https://www.linkedin.com/in/ericbroda/

Eric's Medium: https://medium.com/@ericbroda

Phill's LinkedIn: https://www.linkedin.com/in/redshirts/

Liz's LinkedIn: https://www.linkedin.com/in/elizabeth-negrotti-calloway/

In this episode, guest host Eric Broda, an Executive Consultant in the financial services space (guest of episode #38) facilitated a discussion with Liz Calloway, a Data Governance expert in Financial Services (guest of episode #92), and Phill Radley, Principal Data & AI Strategy Consultant at Thoughtworks. As per usual, all guests were only reflecting their own views.

Scott note: I wanted to share my takeaways rather than trying to reflect the nuance of the panelists' views. This will be the standard for panels going forward.

Scott's Top Takeaways:

1. It’s important to understand that when you decentralize, different aspects can - and should - move at different paces. Your roadmap needs to account for that but it should also take advantage of that. Domains can go at their own pace, including ones looking to quickly drive towards significant data value.
2. Everyone's roadmap, by their inherent nature, will be unique based to the organization's context. Trying to copy-paste from another organization will end badly. That said, there are some pretty core capabilities that all mesh roadmaps should have.
3. A roadmap should point you towards your target endpoint. Of course with data mesh, there isn't really an exact endpoint - you don't stop evolving, much like with microservices - but the idea is the same: how do you want the organization to operate relative to data production and sharing? Set your North Star to guide you.
4. Your roadmap really should have time built in for "socialization". If you don't, then there is no real extra collaboration or communication between teams and everyone executes on what they expect is needed - you optimize at the micro level, not the macro. It's crucial to human cognition to not only be 'doing' but thinking and relating to other people. That's how you can drive sustainable change management.
5. Be very clear on your value proposition for your data mesh journey but don't try to get your budget and set your value proposition as this long-term delivery of value. Look to the incremental value delivery and find value propositions for incremental work. Basically, deliver value along the way and don't tie your budget and value to something that is a massive delivery 3 years down the road.
6. For data mesh, absolutely be prepared to make compromises on when you plan to deliver capabilities. If you aren't prepared to make compromises, you unequivocally are not ready for data mesh.
7. Focus more on business capabilities over purely underlying technical capabilities in your platform roadmap. The tech is the most tangible part but it's also not the thing that really ends up driving value or being overly important to most users.
8. Governance might actually be the most crucial aspect to your roadmap. If you don't map out your planned capabilities around governance, you don't really know when you'll be able to tackle certain types of use cases. And it's also very easy to leave off crucial governance issues to later that will hurt you as you try to scale like policy as code, data product blueprints, and interoperability standards.

Other Important Takeaways (many touch on similar points from different aspects):

1. Try to delay your key architectural decisions until you know more. At some point, you have to make a call but don't try to lock in your architecture right away. However, this will REALLY frustrate many (most?) data engineers because they often care more about the tech than the capabilities.
2. Overall, focus more on what are you trying to do to support and advance the business strategy in your roadmap. Data people want to focus on the data work but it's very easy to diverge from the business strategy. So make sure you are keeping your roadmap aligned to business goals.
3. If you are going to create a roadmap, you should get clear on the definition of your roadmap. What are you actually trying to achieve with your roadmap? What is important to include and what isn't? Don't create a roadmap to create a roadmap - at the start of your process, define the goal around creating the roadmap.
4. A roadmap at the high level is a strong communication tool to drive alignment across many parts of the organization. It is a vision to share and work towards. You will have your top roadmap but you might have roadmaps for aspects of your data mesh like your platform too.
5. The roadmap isn't rigid. The target vision 'destination' is but focus a lot on what is our next 'pit stop', what is our next value delivery instead of trying to plan everything out years in advance.
6. If you're spending too much time to ensure you are 'right', being 'right' might not be actually valuable - you spent too much of your "time, talent, and/or treasure" as Liz said where the return on investment isn't there. Learn to fail fast and iterate to value with smaller bets.
7. Even when it comes to a roadmap, prepare people for one of Zhamak's mantras: think big, start small, move fast.
8. Your roadmap shouldn't be too ambitious and you should roadmap out your organizational changes too. Far too many try to focus purely on the technical and platform aspects. Do not try to escape Conway's Law - look to reflect its impact in your roadmap.
9. Don't focus overly much on exactly where you are going without reflecting on how far you've come. Think about your successes - showcase/celebrate and try to measure them as that will drive further buy-in and momentum. You can highlight the work of the team and inspires others. Make this part of your formal roadmap.
10. You want to think about when you will add your self-service capabilities. You have to make some compromises at first in a data mesh journey, especially around the platform. It can't do everything you want it to on day one or you will have spent far too much time building instead of doing. Be honest about the importance of capabilities and look to automate where you find the most friction.
11. Governance on your roadmap will really set a tone for your journey. Is this about better data processing or about better business outcomes through AI and analytics?
12. Does your definition of minimum viable product for a data product evolve as you mature in your data mesh journey? It's an interesting question to dig deeper into.
13. A simple, compelling value proposition could be 'the work we do to satisfy regulatory compliance should be reused for our analytics and AI. We should drive the cost and complexity down for both by focusing on reuse.'
14. Your roadmap could possibly even include developing a better decisioning process for future work. Focusing on tying business use cases back to data work. Even something like measuring the value of your data work. It won't be great from day one but think about business processes around data work you want to mature.
15. In your platform, potentially err towards building the business capability first and then building out the self-service aspect to take care of that. You need to understand what you actually want to build and an excellent (the best?) way to do that is to get real-world feedback on what are the actual pain-points of putting something into production.
16. It's okay to have either (or both) an org-wide, large-scale transformation roadmap or a much smaller roadmap focused on that incremental value delivery. Both could be a valid approach depending on your organization's situation.
17. Be real with assessing where you are in your data maturation journey, your data governance journey. No one is judging you on that. It's better to be honest to yourself than going for bravado. Again, no one externally is looking at you and especially not at your roadmap :)
18. There are multiple successful and sensible ways to start a data mesh journey. You shouldn't be overly focused on how others are getting going. Yes, learn from them but you can - and should - tweak to your organization. Don't go for easiest, go for most likely to succeed in the short and long run. Yes, easier said than done.
19. Getting your first mesh data product to production is a "momentous occasion" as Eric said. You had to really get a lot of things in place to achieve it.
20. There are a few parallel value delivery tracks in your data mesh journey. There is business value delivery - delivering on the use case. And there is also mesh capability delivery - delivering on the ability to do data mesh. When you first start out, the business value delivery will outstrip the capability delivery but the capability delivery is where you have your long-term value leverage, your ability to scale value delivery.
21. It isn't easy and can sound cheesy, but make sure to build empathy into your roadmap - how do we exchange context with each other and learn what drives value for each other so we can do better work that delivers more business value? Good empathy will lead to stronger communication and collaboration when it's valuable/necessary.
22. If you see something, say something. Over communicate. Think about what others might want to know and let them know, don't only wait for someone to ask.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf