The Importance of Continuous Monitoring in Data Management

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts (most interviews from #32 on) here

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Beth's LinkedIn: https://www.linkedin.com/in/beth-bauer-102449/

Beth's Website: https://posiroi.com/

Harvard Business Review article on 'The 3 Elements of Trust': https://hbr.org/2019/02/the-3-elements-of-trust

In this episode, Scott interviewed Beth Bauer, Founder and CEO, PosiROI. FYI, there are lots of nuggets in this one for people creating a data strategy or trying to tie your data work to value creation.

Beth's ADEPT^2 Framework (covered briefly near the last 10min of the episode): Analytics - Acuity - Data - Decisions - Engagement - Enablement - People - Processes - Technology - Trust

Some key takeaways/thoughts from Beth's point of view, much of which she helped craft:

1. To do data right, we need shared responsibility. There is the technical piece of course but the business aspect is just as important. "…we need to realize that nobody's anything without each other" across the units and enterprise.
2. ?Controversial?: Really good data management can cause some challenges to power structures, especially "how it's always been done" power structures. Try to work with people to give them sight to how they are important in a changed organization.
3. ?Controversial?: Don't think data or digital _transformation_. A transformation is something that completes. This is a journey, an ever evolving journey of improving your data practices.
4. Data fluency is crucial - not just giving people the ability to work with data but the trust, especially in themselves, to leverage and even rely on data. People need guiderails to know where they can safely create insights independently, and where they need to ask for more expert guidance.
5. Trust is made of relationships, judgment, and consistency (from HBR article linked above). Over half (half!) of trust is driven by relationships. If you want people to trust your data, you have to form and build relationships. In the same vein, if data itself is the consistency + judgment, that doesn't even get people half way to trust.
6. It's crucial to think about delivery timelines for data strategy work. There are things that will take much longer and deliver considerable value but you need to break them down into manageable pieces that have incremental value as you deliver. As needs shift, you can react because it isn't a locked long-term strategy. Scott note: that project with a payoff only starting 3yrs down the road, it's always 2.5yrs too late. Deliver value incrementally building to the desired outcome, which can shift.
7. ?Controversial?: Sometimes it's necessary to go from the high-level data strategy vision all the way down into the weeds and vice versa. Oftentimes in data, those weeds really do matter and you should look to connect the strategy to specifics when it's of value.
8. It’s really important to recognize that gaining alternate lenses from outside your usual project team can help, regardless of your data function. There's no shame, and it shouldn't be for everything, but expertise outside your normal team is often of great value. Data is a team sport.
9. "A continuous process of gap analysis is absolutely critical." If you aren’t continually assessing your competencies, your data practice will deteriorate. Unfortunately, we are never done, new competency needs emerge and existing ones can easily atrophy.
10. When people ask for data, it's important to simply ask "what are you going to do with it?" If they can't answer that - what would it change for them? - then you want to figure that out before doing significant work. Scott note: as Alla Hale mentioned in episode #122 "what would having this unlock for you?"
11. To do data well, you need to create a high-level vision and strategy that is flexible and can evolve. And then you need to break down your target milestones and start working out what needs to be worked on when. It's okay to make long-term bets but you need incremental value delivery as well.
12. ?Controversial?: Collaboration in data can be a double-edged sword. It can add significant value but coordination can cause friction and delays. Look for ways to avoid coordination slow-downs so you can deliver value faster and more easily but still collaborate to drive towards mutual benefit/value. You have to trust that others will deliver, give them the room to do so.
13. Data without context, without the proper metadata and owner shaping it, can often be more harmful than beneficial.
14. ?Controversial?: Data people have to get comfortable with giving up control and enabling teams to work autonomously to deliver value. It's important to prevent silos, yes, but you still need to let go of some control to create more value as an organization.
15. Look to the Agile philosophy and find ways to achieve the goals of Agile. Orgs doing Agile often get caught up in the ceremony and in the small picture work. Don't focus on ticket closure achievement, work in a nimble way to build incrementally towards much bigger pictures.
16. Data isn't the point. Data is a driver to do business better. Data for the sake of data is far too common of a trap - avoid it and focus on delivering better business results through data.
17. To do data right, producers and consumers need to be able to talk transparently about goals, expectations, limitations, etc. That goes back to trust - both sides have to be able to have a genuine, high context conversation with strong trust.
18. The business people don't need to know "how the sausage is made" relative to data processing but they do need to know what type of sausage it is, what goes into it, what's the flavor, is it a patty or a link, etc. Scott note: this is sharing the information, not merely the 1s and 0s of data. We need to focus on sharing information, not pure data.
19. You need to build out a data sourcing strategy - both external sourcing and internal sourcing. Internal sourcing then further splits into preparing existing data for usage and creating new sources where you lack the data entirely.
20. It can be easy to lose focus on how data work, especially execution on data work, supports the business strategy. But that's where we constantly see the statistics on data work not meeting expectations - when the work gets disconnected from the business value and business strategy.
21. All parties understanding where information actually comes from, how it was generated or sourced, will drive far more trust in the data.
22. ?Controversial?: Much like it's crazy to set a data strategy that doesn't reflect the business strategy, it's similarly crazy to set a business strategy not backed by data. You might not be overly sophisticated in the data you initially provide to a business strategy team but if they aren't leveraging data, they are missing crucial context and understanding.
23. Minimum viable products (MVPs) are very important to innovation and collaboration. They are "… your first iteration of … did I hear you properly? And are we beginning to create what we needed to together?" But too many try to rush them or overengineer them. Focus on both what minimum and what viable mean. Use this as pilot-testing.

Beth started the conversation with a bit about her background and then got to swinging a bit at current practices :) She talked about the need to have high-level data strategy vision but also be able to understand things in the weeds. Oftentimes those details in the weeds are important to being able to execute. Just don't get lost in them!

Setting a flexible, evolvable strategy and vision is crucial in Beth's view. You need to have a vision of where you want to go before you can figure out what actions you want to take. And then you need to look at milestone goals and break down what needs to be done when. Look to find incremental value delivery instead of putting all your value eggs in the basket that won't pay off until 2-3 years down the road. No one is willing to wait that long and needs will probably shift along the way. It's definitely okay to make long term bets that will require work to deliver value years down the road but focus on delivering value before that too. Priorities will change and so will your understanding of what will deliver the best value. So set out what needs to be done and when and do get going and be ready to reprioritize as you learn!

Beth talked a little about the cost of collaboration. If that is loosely coupled collaboration but not necessarily coordination on most to every step, great. But there is always a friction cost to closely controlled collaboration. Try to avoid work that requires too many dependencies - get aligned and work together towards a common vision. We really need to have trust that other parties can and will deliver. If you need to be working that closely, do you really trust each other?

Agile can be as much of a hindrance as it can be a help in Beth's view. If it's about taking the bigger picture and constantly breaking it down into achievable pieces, great. But if the focus is on completing work instead of driving to bigger picture value - if the work doesn't have an extremely tangible connection to value - you get lost in the cogs of the machine. You need a strategy that can handle how the cogs work but the cogs aren't the point of the machine - what are they actually driving?

Trust is made up of 3 elements according to an HBR article (linked above): relationships, judgment, and consistency. Relationships are over 50% of that trust. Beth believes data is essentially the judgment and the consistency but to get people to use the data we have, the data we can provide, we need to build relationships. You can't just show them the data, it doesn't even get them halfway to trust! Scott note: one comment I make is the difference between someone simply using the data and someone relying or depending on it. It seems like a small differentiation but it's not. One is using the brick as an accent to the building and one is building with the brick as a key element of structural support.

Beth pointed to how while the business folks don't need to know exactly how "the sausage is made" relative to data, they do need to understand more than just 'here's some data for you.' It's about sharing the necessary context. If you want sausage analogies, what flavor, what are the ingredients, what is the shape as in patty versus link, etc. They don't care about the data processing techniques but they should care about - and can gain value from - how was the data transformed from a business perspective. Scott note: this is where I talk about sharing information versus data. Just the 1s and 0s of data have no value without context as Beth said. So embed the context, focus on sharing the context, otherwise it is just values in a more complicated spreadsheet :D

Creating data sourcing strategies - at the micro and macro level - are important in Beth's view. Don’t overly rely on external data, that's costly. What data do you already have internally that you should leverage? What data could you be generating internally that you aren't? Dive into specifics and create a scalable way for lines of business to figure out good paths with sourcing data - internally and externally - going forward. Make sure to look at things from a cross domain lens too and also think about privacy, regulatory, etc.

For Beth, many organizations have trouble keeping the data work aligned to the business strategy. So there needs to be a specific focus on making the work matter, driving to business value. Yes, at the micro level but also on the whole - what business objectives and business outcomes is the data work supporting? Getting "down in the weeds" can also be very helpful, the details do matter as they make it “fit-for-purpose”.

On business strategy and data, Beth echoed the view many other guests have shared that creating a data strategy not aligned to the business is not a smart practice. But almost as egregious is not using data to help power your business strategy but this is extremely commonplace. Data is synthesized knowledge of what is going on in the world, often how the organization is interacting with the world. Why wouldn't you want to leverage that for shaping your strategy?!

In data, Beth has seen the benefit of the MVP (minimum viable product) methodology and they are wonderful if used correctly. However, they are often not used correctly :) Innovation doesn't have a steady timeline - it's messy. MVP timelines are tough so focus on getting to something viable instead of hitting a deadline - and communicate that to stakeholders. MVPs are about making sure you are on the same page and then iterating to better from there.

Beth talked about the need for continuously doing gap analysis with your data and business capabilities. The world is ever changing and new challenges and needs will constantly come up. Plus current capabilities can atrophy. You might be hindered in projects because you need some special capability - especially think legal/regulatory compliance - and you should know that _before_ doing the work :)

Two key questions Beth uses when people ask for data/data work: 1) what are you going to do with this? And 2) How much value do you think this will generate? You don't need to get super specific but people need to at least have a good idea of what the work will unlock and the value of that. If it won't cause any action, why do the work? If the cost outweighs the value, that should be known so you can work to balance that equation by cutting costs and/or finding more value.

For Beth, to do data right, we need shared responsibility. There is the technical piece of course but the business aspect is just as important. "…we need to realize that nobody's anything without each other." We need to drive to address current gaps and we need short, medium, and long-term strategies that drive necessary work in the short, medium, and long-term. Don't get overly focused on the near or the long-term. But it's not just about doing the data work, especially the technical data work. What value and change does this work actually drive?

"And what I found is that largely, a lot of organizations, the challenge is, with really good data management comes really good transparency into how things work. And that really causes pushback on the power structures, and particularly in the 'how it's always been done' power structures. Because if it now points to a way that things can be done better, you start to get into things. Things are happening behind the scenes that have nothing to do with data, and everything to do with people's perceived value of themselves to the organization - without (them) thinking about how they can evolve to actually move from what they're doing today to doing it better." Scott note: it's crucial to help people see how they can move to doing more valuable things - their time of toil is behind them and we can unleash the value creation :)

Quick Tidbits:

In data, there is often a rush to get things done instead of get to automation. It's often - but not always - the right call to slow down to do things right and set yourself up to do them faster/better/more scalably as you move forward.

Lineage, especially for how data was generated and transformed from external sources, is really crucial to increasing trust in that data.

"Data for the sake of data is useless." Scott note: PREACH!

"The digital transformation, to me, is the biggest misnomer and misguidance that we've ever created." A transformation has an end. This is a journey.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or