Data Mesh Radio #135 Iterating Consciously - and Quietly - Towards Data Mesh Capabilities - Interview w/ Balvinder Khurana
Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/
Please Rate and Review us on your podcast app of choice!
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Episode list and links to all available episode transcripts here.
Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.
Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.
Balvinder's LinkedIn: https://www.linkedin.com/in/balvinder-khurana/
In this episode, Scott interviewed Balvinder Khurana, Principal Data Architect at Thoughtworks.
Some key takeaways/thoughts from Balvinder's point of view:
- Data mesh is NOT a silver bullet and not everyone is ready to do data mesh - others have stated that but it's crucial to repeat.
- A data mesh doesn't happen in a vacuum - you need to assess if you are really ready and does it align first to your business strategy and second to your data strategy.
- If you decide to move forward on a data mesh implementation, really consider how you will measure progress and success against business goals.
- To evaluate data mesh appropriately, consider what business value having better data practices would bring to your company and is your company aligned into lines of business or would you need to reorganize your business. Are you prepared to extend your line of business practices to data?
- A common failure pattern in analytics has been not looking at the Intelligence Cycle - changing your operational systems and processes as a result of insights. Don't just generate insights, insights must generate action! Data mesh must avoid this too.
- Even if existing centralized data team setups have significant bottlenecks, data consumers typically eventually get their needed data. Those data consumers can see something like data mesh as a risk - will they still be able to eventually get the data they need? Is eventually getting to faster access to new data worth the perceived risk?
- If you have resistance to data mesh, look at delivering necessary capabilities to your data producers and/or consumers in a small-scale, incremental way and not specifically tying that in their mind to data mesh. Tie those incremental capabilities to business value.
- Look to constantly communicate the what and the why of improvements to your platform to drive engagement. What is the purpose? Why should they care?
- It's easy to fall into the trap of trying to iterate everything constantly. There is a concept of "good enough for now". Don't be focused on getting everything perfect, that juice is not worth the squeeze.
- A good signal to reevaluate your domain boundaries is: a dashboard is still used but the owner no longer uses it. They don't want to own it any more so you need to find a new owner. That probably means changing business boundaries if the original user doesn't use it anymore.
- To drive buy-in for data transformation, especially something like data mesh, you should look to drive it from both top-down and bottom-up if possible. It can work with only one but support from both makes it much, much easier.
- It's okay to start out with lots of manual processes, just be conscious about where you want to automate. Take on tech debt consciously.
- It's easy to get caught up in the trap of trying to optimize for time-to-market on every use case. But you need to balance time-to-market against quality and scope. You don't get to pick all three and might not even get to pick two. Look at what's really crucial.
Balvinder started by discussing the big question: is data mesh right for an organization - right in general and especially right for them right now? You need to start from questioning what are your key strategies - business strategies first before data strategies - and where are you right now. Data mesh isn't your data strategy and implementing data mesh doesn't happen in a vacuum - the real world is messy and ever-changing. So you need to line up your data strategy with your business strategy and then you get to the fun of "okay, how do we measure our success against our business goals?"
Once you have your business goals and how you will measure against them in place, Balvinder believes it's time to look at how can data help you reach your specific business goals and/or measure your success against your business goals. Again, have those business goals in place first. Then you start to look at your data operating model to see if it will help you achieve those goals and measurement. That's when you start to ask if data mesh is right for you.
When evaluating if data mesh is right for you, Balvinder recommends a multi-pronged approach. First is again to look at the value cases, what does your organization want to achieve with data? The second is to ask what challenges are you facing right now? Will data actually help you address those challenges? What are you trying to do with data and how would a more mature data practice help you achieve your goals? And the third is to look at how you are structured, do you have boundaries already set up around domains/lines of business? If these are aligned, then you can start to consider data mesh.
Not paying attention to the Intelligence Cycle is one way Balvinder believes analytical approaches have failed in the past - we need to keep an eye on preventing that failure mode in data mesh. The Intelligence Cycle is about taking information, analyzing it, and then pushing the results of that analysis - either directly or indirectly - back into your operational systems and processes. Essentially, the so what of doing data analysis - is there an actual impact to it? Actionable insights are only valuable if you actually take the action after all.
Balvinder shared the story of an existing client with a centralized data team and an existing data platform: a request for new data would go to the central data team; the data team would reach out to a number of potential data producing teams to try to figure out how to service the request. It took 2-3 months to get data from request to delivery. But, it was a known quantity even if it was slow. They knew who to reach out to. So, they were hesitant to rethink their approach because they can eventually get the data they need. Will it be the same in data mesh? That's a risk many might not be willing to take.
In order to implement data mesh in that kind of environment, to alleviate that perceived risk, you should focus on business continuity per Balvinder. By moving more slowly and only making small incremental changes to the existing platform and ways of working, they were able to move more and more towards data mesh without having to get everyone bought in ahead of time. Adding capabilities and explaining the business value of each capability - instead of trying to sell the data mesh concept as a whole - meant they could continually deliver business value. A thin value slice for each incremental capability with the selling point not being the tech. They created smaller milestones that led to a better path for their client.
And it paid off according to Balvinder - the client team felt empowered to do things for themselves with data. That team wanted to launch a new KPI and they already had all they needed to do it with the slow feeding method of adding capabilities and resources through a better platform. So they didn't need to go to the central team and quickly launched their new KPI.
Constant communication was crucial for Balvinder and team to drive engagement and buy-in. And it's important to not communicate just once as things will inevitably evolve as the real world changes. And it means you can deliver continuous improvements instead of trying for a big bang approach. Your domains will continuously change, your platform will need to evolve, etc. There is a concept of too much iteration and change as well - it's okay to assess if something is "good enough for now". You don't need things to be perfect, look to change where there is the most benefit.
For Balvinder, it's really easy to get overly focused on "I want this in production yesterday." But, when a data consumer says that, then they must understand they need to compromise on quality and scope. It's not necessarily even you get to pick two, there are aspects of all three and if you focus on time-to-market, it will mean it's not the quality they want and the scope needs to be limited. So what really matters most to the business? And if the answer is always time-to-market, Scott believes you might need to talk to your data consumers about what really matters.
Balvinder wrapped up on a theme many guests have touched on: data mesh is not some silver bullet. It won't solve all your challenges. You need to really think about if it's right for you. And there are many companies that aren't ready for data mesh. And that's okay too!
A few quick tidbits:
A good signal to reevaluate your domain boundaries is: a dashboard is still used but the owner no longer uses it. They signal they don't want to own it any more so it needs a new owner. That probably means changing boundaries because the business need that drove the creation of the dashboard is no longer with the team that created that dashboard.
To drive buy-in for data transformation, especially something like data mesh, you should look to drive it from both top-down and bottom-up if possible. It can work with only one but support from both makes it much, much easier. Having management support that the data initiatives/strategy align to the business strategy is important but so is buy-in from the people actually doing the work :)
It's okay to start out with lots of manual processes, just be conscious about where you want to automate. Take on tech debt consciously. When it's time to improve that capability in the platform, invest in it. But don't try to build everything ahead of time.
Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf