Navigating Analysis Paralysis and Scaling Solutions

Please Rate and Review us on your podcast app of choice!

Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Christoph's LinkedIn: https://www.linkedin.com/in/christoph-spohr-2a33b619b/

In this episode, Scott interviewed Christoph Spohr, Lead Architect of Big Data Platforms and the Product Owner of Data Mesh at Volkswagen Group. To be clear, Christoph was only representing his own views on the episode.

Quick Note: apologies that Scott's audio is a bit weird, he had yet to build his makeshift sound studio in the Netherlands.

Some key takeaways/thoughts from Christoph's point of view:

1. "There are not solutions, only trade-offs." Understand that nothing is perfect.
2. !Controversial!: Your biggest risk to a use case is choosing the wrong abstraction, the wrong architectural paradigm. When working with a domain, look for a way to achieve the required result that is as close to their existing way of working as possible. Scott note: this can become challenging on finding reusable ways of working if you go too far but greatly smooths domain-level adoption. It's all a tradeoff 😜
3. The second biggest risk is do stakeholders really feel and acknowledge the pain the existing approach/system is causing and want to move? The third biggest risk is about promising things you cannot deliver - obviously look to avoid that.
4. ?Controversial?: A really key aspect of doing data mesh well is having and keeping perspective: doing data mesh is more about "aligning people on a shared vision and not getting lost in the overall process." Don't get so bogged down in exact implementation details if possible.
5. Keep people informed and aligned and start your conversations on clarifying what people believe/understand about data mesh so you aren't talking past each other. For many people in your organization, their understanding of data mesh will be shaped by vendor content/conversations. You need to check alignment many times as it can drift.
6. Advice to former data mesh self: Really focus on keeping that communication between all parties going and making sure people are aligned on the approach, complexities, trade-offs, and risks. And don't think people automatically stay aligned, stay on top of the communication and relationships.
7. In a large organization, if you want to implement data mesh, find sweet spots for it to succeed. Because you will have to make compromises/trade-offs. Everything in data and IT is about finding the right compromises but don't put yourself in a place where your compromises fully undermine your data mesh vision.
8. When putting together your enterprise-level vision for data mesh, you need to break it down into two parts: A) how would it work with and serve existing business processes and B) how to approach the IT architecture and assessing/acquiring the necessary capabilities to do data mesh.
9. Data mesh is one of the most - if not the most - complex data architectures and you need to have a strong understanding of the complexities and not underestimate them. Otherwise, scalability will likely be a significant challenge.
10. Data mesh does not mean all your data problems go away. As Zhamak says, it also creates new ones.
11. It's important to listen to stakeholders' challenges and deliver a solution that addresses those challenges. If you try to deliver to their exact wants, those often aren't scalable ways to address their challenges. Focus to the end result, not what they _say_ they want.
12. When it comes to data mesh, make sure stakeholders understand that there is a cultural change component. If they want things to work at the speed and scale data mesh can provide if done well, approaches and hearts/minds have to change. Data mesh is not a magic wand to wave at your data to make your approaches/solutions scalable.
13. ?Controversial?: Your IT people need to work on their communication skills. And watch out for IT/data team members getting a big head about what they can do, that can create lots of challenges.
14. Plant your data mesh seeds early - have conversations with stakeholders, even when they aren't ready to take advantage of the data mesh approach/your implementation yet. They will be much more likely to come to you for help when they are ready and eager.
15. Try to understand your senior leaders' IT systems background and their favorite - or at least known - approaches. When you frame data work in a way they understand, even if some of the implementation aspects change, they are much more likely to be in favor of it.
16. ?Controversial?: Instead of going to the business leaders, look to talk to the technical people within a domain first to find out the business challenges at the technical level. That way, you can bring the business leaders an idea of an approach that is more fully formed. Scott note: you can take this too far quite easily and try to come in with only a technical solution to the challenge.
17. ?Controversial?: People think data platforms are mostly about doing more and more complex analysis on data but that's rarely the use cases that add the most value. Scott note: strongly agree, reducing friction to leveraging data is rarely about the most complex data and analytics practices.
18. "[My] loyalty to data mesh ends when complexity gets too big." Scott note: take this one to heart. Data mesh for the sake of data mesh helps no one - except maybe a resume 😅
19. Don't get bogged down by analysis paralysis. Look to work in a simple and speedy manner - if you have to backtrack, it is almost certainly still going to be quicker than trying to answer every question ahead of doing the work.

Christoph started the conversation with some perspective into the IT/data landscape at Volkswagen where they have 100Ks of employees and 100s of petabytes of data. So, when you think about what good data mesh practices look like in that environment, at that scale, it's much more about "aligning people on a shared vision and not getting lost in the overall process." And it's easy to lose alignment because so many people are getting their understanding about data mesh in general - or at least aspects of data mesh - from vendors. Those definitions are often very different, especially around how much is centralized or decentralized. So look to start your conversations with colleagues by clarifying everyone's understanding of data mesh or a specific aspect of data mesh to stay on the same page. Your understanding will change over time and so will theirs.

In data and IT work, at the end of the day, everything is about compromises and tradeoffs for Christoph. Nothing is the "right" answer, it is about the search for the least bad answer. When you are looking at data mesh in a very large organization like Volkswagen, he recommends looking for your sweet spots - there are places data mesh could thrive and places where it's not a good fit. Don't get too bogged down in trying to go organization wide early on. You couldn't support that anyway!

Christoph recommends before getting too deep into your data mesh implementation strategy, break down your high-level vision into two pieces. The first is: how would this align to and serve the existing business processes? You need a business side of your vision - what are we doing data mesh for? You also won't be able to change the way the organization works easily so find ways to align to how things already work. The second is the IT architecture and the necessary capabilities to actually pull off doing data mesh well. How do you get into a data-driven mindset? What are the kinds of technical implementation details you will need to accommodate - not even the use cases, just what types of domains and sources. How will you manage risk, how will you make sure things can scale?

Choosing the wrong abstraction, the wrong architectural paradigm is the biggest risk to a use case in Christoph's view. Each team has a different way of working and you should look to, still within the framework of data mesh, find a way of working that achieves the target results but is as close to their existing way of working as possible. This will likely smooth adoption friction and prevent some misunderstandings. His next two biggest risks were first, make sure you have stakeholders that really feel the pain of the existing approach to the use case and systems and are willing to move. If there isn't a desire to move, it's difficult to drag them along and succeed. The other - the third biggest - risk is about don't over-promise; make sure what you say you will deliver is realistic.

For Christoph, "there are no solutions, only trade-offs." When discussing approaches, let people know the trade-offs you are making because nothing will be perfect. That honesty will resonate well with most people.

In Volkswagen's journey specifically, Christoph expected the biggest pain point for customers to be that data was in silos but it wasn't - they were more concerned with fine-grained access control. So when he talks to stakeholders, he makes sure to understand their pain first instead of trying to pitch the approach they've landed on with data mesh. But it's also important to not ask people what they want in his experience, they will try to just add scaling to their existing approach when a new approach is often necessary. So, listen for the pain, pitch to the pain, but don't pitch to their exact desired approach if it's not a scalable solution.

When asked about advice to his former data mesh self, Christoph circled back to the challenges around communication. You need to constantly be staying in touch with people, aligning on vision and sharing trade-offs. You also need to focus on what are people's pains and look to deliver something scalable to that pain rather than asking them what they want - they often want their current approach to magically scale. You have to partner with them and make sure they understand that cultural change will be necessary to accomplish their goals - this isn't a magic mesh wand. And he also recommends watching out for any of the IT/data team getting too big of a head because they can do really interesting things with data.

Christoph recommends a few things for getting more buy-in for your data mesh implementation in a large organization. One is to plant your seeds by having lots of conversations about what you are trying to achieve because when people are ready, they will come to you. A second is to talk to a lot of people to understand where big challenges/opportunities are in the organization. That way, you can try to focus a bit more on high-value use cases. And the third is try to frame your approach and choose an abstraction that matches the senior leader/stakeholders. If they like one approach, try to frame things - and hopefully develop your approach - in a way that works well for them so you aren't trying to drive buy-in for the approach as well.

Volkswagen is potentially a different kind of organization to many others out there. In Christoph's experience, as a hierarchical organization, it often makes sense to talk to the technical people within a domain first to assess what challenges there are and come to the senior leaders with something more fully formed and targeting their actual business challenges rather than asking them to share their data related challenges. Be opportunistic in selecting use cases to go after. And he is also not loyal to the concept of data mesh - if it gets too complex at scale, he will look for something else.

As to ways of working, Christoph recommends to go for simple and speedy because even if you head down a path that isn't working, it will still likely be quicker to pivot and head in a new direction than if you had gotten bogged down with analysis paralysis. Look to converse with other experts in the industry on your journey - Christoph was unlucky in finding people who had done data mesh at Volkswagen scale but that's not the case for most organizations.

Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf