Data Mesh Radio #276 Making Self-Service Actually Work Well Safely - Interview w/ Kate Carruthers
Please Rate and Review us on your podcast app of choice!
Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Episode list and links to all available episode transcripts here.
Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn.
Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here.
Kate's LinkedIn: https://www.linkedin.com/in/katecarruthers/
Kate's 'Data Revolution' Podcast: https://datarevolution.tech/
In this episode, Scott interviewed Kate Carruthers, Head Of Business Intelligence at the UNSW AI Institute and Chief Data & Insights Officer at UNSW (University of New South Wales). To be clear, she was only representing her own views on the episode.
UNSW is not currently implementing data mesh but are preparing to be able to do so. This is a great lesson in building up the capabilities to move forward towards your goals but not rush.
Some key takeaways/thoughts from Kate's point of view:
- Universities can teach us some really interesting perspectives on self-serve. Because universities are such complex organizations and so many departments are involved in deep investigations in very specific areas, they really are the only domain experts. So enabling them to even just own their own data can be very challenging, let alone helping them share with others safely.
- Relatedly, each academic researcher is essentially a micro-domain themselves with their own ways of working. That just adds to the need to enable freedom in ways of working but still "keep them safe." Scott note: safety was a key theme of the conversation
- "At the end of the day, data mesh is about controlling the bits that you need to control, and giving people the freedom to do what they need to do, safely."
- "Technology is kind of the least of your problems." When it comes to data, be prepared to start with some people not even recognizing there is a problem with the current ways of working or a need to improve. Connect their pain to data immaturity to win them over.
- The best way to win people over is show, don't tell. Show them the power of self-service instead of pitch them on it. Get a PoC going and get people to tangibly see - and hopefully soon touch - your self-service capabilities early.
- Always look to anchor your data work - especially things like platform work - to a business need. How will doing the work impact the business? Why is it important to do and to do now?
- When tying your data work to the overall business strategy for your organization, do NOT forget the people aspect. The relationships matter. Your work on the data team definitely isn't only about technical execution.
- ?Controversial?: Build a culture around data that is as focused on building human relationships as it is on building data pipelines and platforms.
- ?Controversial?: To share personal/sensitive information - e.g. PII - a producer should justify why it's appropriate and a data controller should review that. Keep humans in the loop.
- Giving data owners (UNSW calls them data controllers) a say in how their data is actually used can get them more excited to share their data. It isn't a silver bullet to data sharing incentivization but it adds value to them.
- Good conversations about access to sensitive data shouldn't be yes/no. They are about getting to what is acceptable and maximizing value within that framing. Get people to share what they are trying to accomplish and partner to best achieve it!
- Invest in business analysts. They are your front-line to figuring out how to proceed around data and generate value. You need people who can speak business and data simultaneously to drive to great outcomes.
- Find ways to prevent data puddles, especially places where people are copying data and not securing it well.
- "People overestimate the power of making change really fast, and underestimate the power of … sustained incremental change."
- Give people a mental map for change. It removes the fear of the change and lets them lean in. You are creating change with and through them instead of pushing change on them.
- ?Controversial?: ChatGPT and other GenAI can actually be a great benefit to education. We have to lean in to it as it's not as though students won't have access to these tools in their work life. So getting them to still learn but leveraging better tools is essential to their progress.
Kate started out with a bit about the catalyst for her current data journey towards data mesh. About 10 years ago, she saw that universities and especially UNSW were going to "undergo a very big digital transformation and that data would underpin it as an organization. [So] we would need to be on top of our data if we were going to be able to ride that wave." She also gave some color on what running a data office at a university entails. At UNSW, it's split into three general areas of administration, learning + teaching, and pure research.
There are some major challenges when it comes to providing data capabilities - especially self-service - to the academic research arm of a university according to Kate. They all have their own ways of working and want - demand? - freedom to work the way they want. Yet, the data team's job is also to "keep them safe." That safety has many facets as well. And the research capabilities of a university can mean some truly world-changing interdisciplinary collaboration. But that only happens if the teams can actually, you know, collaborate 😅
When it comes to the non-research area, Kate believes data mesh is an even better fit. "At the end of the day, data mesh is about controlling the bits that you need to control, and giving people the freedom to do what they need to do, safely."
As many guests have noted, Kate believes when it comes to your organization's data journey, "technology is kind of the least of your problems." It’s about people and often even getting them to recognize the problem with their ways of working and how better data maturity will help alleviate their problems. It's not just the data itself but their understanding and relationship to data.
Kate and team built a quick cloud warehouse PoC that showed people the ability to onboard new data sources in weeks instead of taking up to six months. Showing them instead of simply telling them really won people over. People could connect moving to a cloud data warehouse to business benefits. They also anchored it all to business needs. Yes, rebuilding their architecture to move to the cloud was going to be work but it meant speed to new data use cases and easier management.
When Kate was working to tie her team's work to the overall business strategy, she remained focused on the human relationships and people aspects of doing business. She really recommends building relationships with "customers" of your data work because then they feel comfortable to come to you with more types of problems and challenges. And sometimes that kind of culture/approach isn't for everyone and that's okay. If people aren't willing to treat customers as people, they aren't right for her team.
When asked about her frequent use of the word "safe", Kate talked about keeping people from misusing data or even misusing the trust people who provided that data - e.g. the students at UNSW - gave the organization. Anytime someone wants to share sensitive information like PII, there is a data controller that needs to review the justification. Keeping that human in the loop means there is a real understanding and consideration of 'is this okay?' On the flip side, the team has been proactive in sharing information that someone should have access to, e.g. a professor being able to know who is in their class and being able to contact them.
Kate mentioned that when they implemented the data controller review, the data producers were much happier. Previously, they had no real say in how their data was used but now, they are listened to. It also strengthened relationships because consumers had to actually collaborate with producers to get access to their data. It's creating interesting conversations and people can get more creative around data to achieve their goals with more data safety. And her investment in hiring a bunch of business analysts has created some great value leverage points.
Going back to keeping people and data safe, Kate talked about their struggles with data puddles - where people are copying data into lots of areas instead of accessing the data where it is. And they aren't securing that data well, which leads to more challenges and potential issues. But it's still a process to give people all the access they need and make that copying data less attractive. As like many areas, it's a work in progress 😅
Kate sees the attractiveness of moving fast but believes people need to focus more on sustained incremental change, that they overestimate the value of the former and underestimate the value of the latter. It's similar to transformation versus a change that will revert. Fast changes are far less likely to stick or even work. And people feel less of the suddenness and fight against it far less if at all when it is gradual incremental progress.
Another point Kate emphasized was that people need a mental map for change. If they don't understand what is changing and why, they will inherently fight back, even if the change is good for them. It's simply human nature to not want change. So take away the fear of change to make it easier for people. Basically create change with and through people instead of pushing change on them whether they want it or not 😅
The conversation wrapped up around GenAI, especially because Kate is involved in the UNSW AI Institute. She is seeing the open source large-language models (LLMs) improving at a rapid pace, sometimes multiple times a day. And there is a lot of promise even if things are early days. At UNSW, they are figuring out good ways to leverage GenAI in education instead of trying to fight against it like some math teachers did against calculators. It's here to stay so they have to adapt and adopt.
Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about
Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf