Data Mesh Radio #236 Driving Buy-in For Decomposing the Monolith; and Then Actually Doing It - Interview w/ Brenda Contreras
Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/
Please Rate and Review us on your podcast app of choice!
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Episode list and links to all available episode transcripts here.
Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn if you want to chat data mesh.
Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.
Brenda's LinkedIn: https://www.linkedin.com/in/brenda-contreras-9649a47/
In this episode, Scott interviewed Brenda Contreras, VP of Engineering and Architecture at Self Financial.
Some key takeaways/thoughts from Brenda's point of view:
- "Iterate small and sell your solutions on a practical level."
- It's kind of funny how often people in tech try to skip the communication. If you really align on communication and understanding, your business partners are far more likely to empower you to drive business value for them through engineering and data work.
- ?Controversial?: As an engineering/data leader, don't dictate: set the vision, explain the vision to business partners, but try to let your technical team leverage patterns that will work for them instead of only your favorite way.
- Similarly, make sure your team understands which aspects of target outcomes drive value and why. They might have an approach you didn't expect but if they aren't focused on the key aspects of the outcome, even amazing feats of engineering won't create value if it's not tied to business needs.
- Fail fast is very important to doing microservices right. How can we learn to adopt it in data and AI? "We need we need to be … able to experiment more, we need to be more flexible" to really drive to business value quicker and easier.
- Before you start to decompose anything, it's crucial to understand what you already have. That can sound a bit obvious but if you start trying to do the work before understanding the 'before' picture, getting to a good 'after' picture is going to be very hard.
- You have to understand the landscape of your systems but it's equally important to understand how those systems power the actual business. Not just services or products, but the business. What matters and why? Talk with and carefully listen to your business partners to find those.
- It's crucial to align with your business counterparts on big picture visions and then what aspect of the big picture you will address and get clear on why and when. This can feel obvious but it's often overlooked. Over-communicate, share your vision often and far and wide.
- Your business line leaders don't care about the how of tech - most of the time at least. They care how can you support them in tackling their goals and challenges. Make your communication about changes focused on that. They don't care how the sausage is made.
- It's absolutely valid, even in a data mesh or microservices journey, to have a "sustain bucket". Things don't need to change for the sake of change. 'Legacy' doesn't have to mean bad/a headache.
- When you start to talk about breaking down the monolith internally, it's important to sell people on the reality: managing smaller systems is easier. Having one group that owns a set of systems - and that it's clearly defined - makes it much easier to find the information you need. Microservices isn't easy but it does make things much easier to manage at the micro level.
- Save talking deep tech or data for the people doing the tech or data work - when you talk to business partners, abstract that for them. They don't care about the particulars of your system for sending emails - they care that the emails are reliably sent and tied to the business aspect.
- When you first start moving towards microservices, look to start setting up your domains around your products. Then you can start to break your systems down into microservices from there. You should think about data mesh the same - don't start building the data products without knowing the exact owner and why they own it.
- Don't try to toss out the old in favor of the new. Look to net new on your new platform or approach. Just because something has been around for a while doesn't mean it isn't valuable as is. Look for where changes will make a big positive impact.
- Be very ambitious in your vision for where you could go. But take it at a reasonable pace - don't be in so much of a rush that you create more challenges than you address.
- You can sell the value of experimentation at the experiment level. It's hard to just get an overarching experimentation budget but constantly showing the value of fast experimentation will make it easier to get the time to do more experimentation in the future.
Brenda started with a little about her history, mostly on the app development and software engineering side before moving to Self where she's added data/analytics to her architectural focus. She learned a lot working in the office of the CTO at Charles Schwab about how to decompose a monolith and not become overly rigid around the philosophy of microservices at the same time - how do you still have shared services? How do split into product domains? How do you make sure everything plays nicely together internally and for the customer?
That learning about how to share data well on the operational plane is serving Brenda well in general at Self. When she first arrived, creating an inventory of what already existed and how it worked together was crucial to start decomposing appropriately. What were the systems yes but especially how they powered the actual business. What really matters and why? What are the key strategic initiatives and how can you positively impact them by improving systems? A 99% improvement in latency on something that drives no change to how people use systems is wasted if impressive work. It's okay for things to live in a "sustain bucket" as you focus on key business drivers. And you can only really find those key drivers by listening to your business partners.
When Brenda started to consider where to start with decomposing the operational systems at Self, she first looked to where they might "run out of runway". What were the systems that were likely to cause trouble driven by growth? Because you don't want to have to tell your business partners they have to stop growth 😅
A key aspect of decomposing a monolith is getting buy-in. Brenda had a lot of success with showing people the benefits of smaller systems that are easier to manage. Deploys are easier. Maintaining a smaller database is easier. It's easier to figure out who owns what - if someone needs information, they know who to go to. Etc. And extracting the information and then explaining your plan for change should be done in the language of the business. They don't care if you are switching your backend to Cassandra, they care about what impact changes will have on the business and what changes for them.
Brenda recommends that when you start to look at decomposing a monolith into microservices - so on the application side - you should start by breaking into domains first. Without clear owners and sets of microservices, you start to split things off without a clear forward path. Once you have the domains identified, you can start to move necessary capabilities into your microservices.
Differentiating baby and bathwater - what should get tossed out and what should be kept - is crucial in Brenda's mind when you are doing any kind of transformation. There are existing solutions that you don't need to migrate immediately because they're fine as is. Again, don't look to make changes for the sake of changing something. Look to net new use cases as you start to decompose and eventually those legacy use cases will get reimagined and replaced.
It's very easy to focus on the wrong things in Brenda's experience - instead, focus on the conversations with business partners and work backwards towards what really matters. Then the work follows from that. E.g. in an acquisition, integrating systems and/or migrating the acquired company to your systems seems to be a major focus instead of integrating what will drive value and keeping things again in that "sustain bucket".
Brenda's secret sauce to aligning with business partners comes back again to communication and understanding the big picture of the organization, how their part of the organization fits in, and what are their priorities. Essentially, it's taking the vision for where we want to go and breaking that into manageable projects to move forward on. Always be communicating priorities and timelines and working with people to realign at the small and large scale vision level. And, a biggie is to _really_ listen to your business partners and C-level execs when they talk challenges.
"We need to be … able to experiment more, we need to be more flexible," Brenda said relative to figuring out how to drive more business value around data. The fast fail approach in software is crucial to getting to business value fast - how can we adopt that in data? Partly, it's again going back to communicating well with your business leaders through good storytelling. She said, "iterate small and sell … your solutions on a practical level."
In wrapping up, Brenda again emphasized the importance of good communication and collaboration with your business partners. It's so easy in data and tech in general to let that drop to focus elsewhere. But when you are all at least reading from the same book, even if there is some disagreement in strategy, people can at least make informed decisions. Don't try to skip the communication with your business partners!
Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about
Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf