Transitioning from Decentralization to Data Mesh

Please Rate and Review us on your podcast app of choice!

Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn.

Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here.

Nailya's LinkedIn: https://www.linkedin.com/in/nailya-sabirzyanova-5b724310b/

In this episode, Scott interviewed Nailya Sabirzyanova, Digitalization Manager at DHL and a PhD Candidate around data architecture and data driven transformation. To be clear, she was only representing her own views on the episode.

Some key takeaways/thoughts from Nailya's point of view:

1. When it came to microservices and digital transformation, we aligned our application and business architectures. Now, we have to align our application, business, and data architectures if we want to really move towards being data-driven.
2. To do data transformation well, you must align it to your application architecture transformation. Otherwise, you have two things transforming simultaneously but not in conjunction.
3. It's crucial to involve business counterparts in your data architectural transformation. They know the business architecture best and the data architecture is there to best serve the business. That is a prerequisite to enable continuous business value-generation from the transformation.
4. Re a transformation, ask two simple questions to your stakeholders: What should this transformation enable? How should we enable it? It will give them a chance to share their pain points and their ideas on how to address them. The business stakeholders know their business problems better than the data people 😅
5. Your approach to data mesh, at the start and throughout your journey, MUST be adapted to your organization's organizational model and ways of working. Everyone starts from completely different places.
6. Data mesh won't work if you overly decentralize. You must find your balances between centralization and decentralization yourself.
7. ?Controversial?: Historically, teams were charged for data work and resources but with something like data mesh, they can manage their data and data costs far more efficiently. Framework processes, tools, and skills help teams to identify which data is valuable for their own or other domains and requires investment, and which data or data processing operations are redundant, and thus, a source of savings.
8. ?Controversial?: You should consider two phases of your early data mesh implementation: “foundation” - enabling teams to own their data by building corresponding teams, processes, and tools and “operationalization and scaling” - enabling/incentivizing them to share their data well with others. They have overlap but if you don't focus on enabling to own data for themselves, you may have trouble incentivizing them to even own their own data let alone share it.
9. To drive incentivization and prioritization well to do something like data mesh - or really most large-scale transformations - you need top-down support from the highest levels in the organization.
10. ?Controversial?: In highly regulated industries, you will have domains that already have very strong governance practices. Focus on enabling them to safely manage their data within a new framework, rather than trying to change their ways of working.
11. Relatedly, focus on the frameworks and guidelines as well as the tooling to enable those domains that aren't nearly as advanced in their governance.
12. If you are looking to federate/decentralize to all domains at the same time, consider how you leverage central committees. If you don't have someone helping guide people towards some degree of consistency, you can't find repeatable scaling patterns/best practices and are likely to create data silos.
13. The level of importance your company places on your data transformation - mesh or otherwise - should determine how you combine it with your digital transformation. It might be under it as part of digital transformation, entirely separate but at a peer level, etc. But they should get aligned no matter what to have the best impact.
14. Everyone must understand that data mesh is journey. You will learn and adjust along the way.
15. Getting budget for your data transformation / data mesh journey is probably more political than many expect. The easiest way to get budget is high-level management attention. Scott note: but of course, it can be hard to get that buy-in first 😅

Nailya started the conversation on the need for application, business, and data architectures to all be aligned. She gave some history about how application and business architecture were brought into alignment when we started with microservices and digital transformation. But now we have to add data into the mix which makes things even more difficult. We need both the application and data architectures to be designed to specifically support the business goals.

When it comes to actually transforming your data architecture, Nailya believes the transformation should be led from the business side where possible. At the very least, the business side should be involved. They understand the business needs best so they can help direct the transformation to serve those needs. Great data work that doesn't support the business needs is often just a well-designed money pit 😅 You obviously need the strong data expertise but a transformation led exclusively by the data team is far less likely to align to business goals and priorities.

In a successful past digital and data transformation, Nailya used two simple questions to the key business stakeholders: What should this transformation enable? And how should we enable it? It gave the business stakeholders a chance to fully lay out their pain points as well as some ideas how to address them. That way, the team had a very broad perspective and could come back to each of the stakeholders with solutions that worked for them somewhat tailored to their needs and thoughts. You need repeatable patterns/approaches but you also need people to feel seen and heard in order to drive buy-in where you address their specific pains and ideas.

When asked about adapting data mesh to an organization's specific challenges, Nailya pointed to how every culture is so different and you need to take into account how people internally exchange information and work together as you design how you want to go forward. Overly decentralizing - so not doing any federation - or decentralizing too quickly won't work well. You have to find your balance between centralization and decentralization throughout your journey.

One interesting buy-in point Nailya mentioned was cost control over data work. Because teams have traditionally been charged for data resources and work by central data teams, they were not as involved in managing costs. Data mesh empowers business teams with tools to control cost-effectiveness of their data, and thus they can identify easier which data is valuable for their business and requires investment, and which data or data processing operations are redundant, and thus, a source of savings. They can see it as a chance to do things better and align better on what work is worth doing - the central data team might have done work that the domain doesn't see as valuable when really considering it more deeply. At first, it might be only for their internal-facing to the domain data work before we can get them bought in that they are now responsible for also sharing their data with other domains.

Nailya talked about her experience with a large-scale data mesh implementation. They focused on first enabling teams to own their own data. So again, giving them the chance to gain transparency to their most valuable data as well as define, align, and prioritize their data initiatives. Then, they started to work to incentivize and better enable them to share their data with the rest of the organization identifying new use cases and data customers. This may delay the biggest benefits of data mesh - high-quality, reusable data across domain boundaries - but it does mean that teams aren't struggling to own their data at the same time as learning to share it with others; this also helps with the incentivization challenge as they can take advantage of their data first for themselves before being asked to focus on sharing it with other domains.

Data governance in data mesh will - unsurprisingly - be hard for every organization in Nailya's view. If there are domains that already know how to handle their data well, work to enable them to better share their information but also don't try to push them towards central ways of working. If they can safely secure and share their data in a way the rest of the organization can consume it, don't get in their way. But you should also look to create frameworks and standards for those that aren't as mature to help guide them along. Scott note: this is an adjustment for those that are already somewhat decentralized with their data work. Again, adjust for your circumstances!

Nailya also recommends central committees to ensure teams are meeting some degree of conceptual consistency and also technical/architectural consistency. That way, you can really find your scalability patterns and best practices. Scott note: if you decentralize/federate all at once, this might be the best bet. But many - most? - are going domain by domain so this function is already embedded in an enabling team.

In her experience, Nailya believes that aligning your data and digital transformation is very important to be able to succeed at data mesh. Again, you need to align the application and data architectures with the business architecture. Really take stock - is your data transformation part of your overall digital transformation, are they at the same level and should be partnered, etc.?

Nailya again circled back to engaging with your business partners/stakeholders to have them help design your transformation efforts. They will lean in from feeling seen and heard but also, they know the most critical pain points best and can help direct where you should focus first.

The conversation finished up around getting and maintaining a budget around your data mesh implementation. For Nailya, this is typically far more political than many might expect. But if you have the proper top-down support and management attention/buy-in, you should at least be able to get going. You have to show value along the way but that should be part of the prerequisite to start: what value are you trying to capture and how will you measure it?

Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf