Experiencing Data w/ Brian T. O’Neill (UX for AI Data Products, SAAS Analytics, Data Product Management)

Brian T. O’Neill, data product development expert, discusses 15 ways to increase user adoption of data products. Highlighted topics include designing behavior change, exposing team members to end users, changing funding models, defining intended benefits and outcomes, and approaching data product creation as a user experience.

Nick Zervoudis, Data Product Manager at CKDelta, discusses his approach to developing internal and external data products, the UX design course he took, thoughts on dashboard design, common mistakes made by data product teams, and how he manages imposter syndrome in his career.

Marnix from Coolblue shares the story of why he threw out 18 months of data science work and shifted to a human-centered, product approach. He discusses the impact on his team, stakeholders, and his personal career. The podcast also explores the challenges of building a data product culture and improving data literacy.

Vishal Singh, Head of Data Products at Starburst and co-author of Data Products for Dummies, discusses the importance of defining data products and shares his definition. He explains tactics for gathering feedback, the challenges of user feedback during iteration, and the danger of sunk cost bias. Vishal also describes the role of a Data Product Manager and highlights the value of self-service usability and good design for data products.

Jonathan Cairns-Terry, Head of Insight Products at the Care Quality Commission, shares insights on their shift to a product-led approach. Topics discussed include the benefits of UX research, investing in a designer, and recent successes. They also talk about the transition from math to product and the importance of understanding user needs and outcomes.

Anthony Deighton, General Manager of Data Products at Tamr, discusses the importance of focusing on customer needs when designing data products. He explores the challenges of building a product for creating better internal data products and the evolution of data product management. Anthony also shares his definition of a data product and whether Tamr qualifies as one. He emphasizes the importance of outcomes and benefits over features and discusses challenges with metrics like Propensity to Churn.

Vera Liao, Principal Researcher at Microsoft, discusses the importance of human-centered approach in rendering model explainability within a UI. She shares insights on why example-based explanations tend to out-perform feature-based ones and why traditional XAI methods may not be the solution for every explainability problem. Vera advocates for qualitative research in tandem with model work to improve outcomes and highlights the challenges of responsible AI.

In this podcast, the speaker introduces the PiCAA Framework for generating ML/AI use cases from a UX perspective. They emphasize the importance of brainstorming ideas, considering human factors, and involving cross-functional teams. The Pico and PiCAA frameworks are discussed, along with examples of AI use cases. The risks of automation and the importance of a human-centered approach are also highlighted.

Today I’m wrapping up my observations from the CDOIQ Symposium and sharing what’s new in the world of data. I was only able to attend a handful of sessions, but they were primarily ones tied to the topic of data products, which, of course, brings us to “What’s a data product?” During this episode, I cover some of what I’ve been hearing about the definition of this word, and I also share my revised v2 definition. I also walk through some of the questions that CDOs and fellow attendees were asking at the sessions I went to and a few reactions to those questions. Finally, I announce an exciting development on the launch of the Data Product Leadership Community.   Highlights/ Skip to:   Brian introduces the topic for this episode, including his wrap-up of the CDOIQ Symposium (00:29) The general impressions Brian heard at the Symposium, including a focus on people & culture and an emphasis on data products (01:51) The three main areas the definition of a data product covers according to Brian’s observations (04:43) Brian describes how companies are looking for successful data product development models to follow and explores where new Data Product Managers are coming from (07:17) A methodology that Brian feels leads to a successful data product team (10:14) How Brian feels digital-native folks see the world of data products differently (11:29) The topic of Data Mesh and Human-Centered Design and how it came up in two presentations at the CDOIQ Symposium (13:24) The rarity of design and UX being talked about at data conferences, and why Brian feels that is the case (15:24) Brian’s current definition of a data product and how it’s evolved from his V1 definition (18:43) Brian lists the main questions that were being asked at CDOIQ sessions he attended around data products (22:19) Where to find answers to many of the questions being asked about data products and an update on the Data Product Leader Community that he will launch in August 2023 (24:28) Quotes from Today’s Episode “I think generally what’s happening is the technology continues to evolve, I think it generally continues to get easier, and all of the people and cultural parts and the change management and all of that, that problem just persists no matter what. And so, I guess the question is, what are we going to do about it?” — Brian T. O’Neill (03:11) “The feeling I got from the questions [at the CDOIQ Symposium], … and particularly the ones that were talking about the role of data product management and the value of these things was, it’s like they’re looking for a recipe to follow.” — Brian T. O’Neill (07:17) “My guess is people are just kind of reading up about it, self-training a bit, and trying to learn how to do product on their own. I think that’s how you learn how to do stuff is largely through trial and error. You can read books, you can do all that stuff, but beginning to do it is part of it.” — Brian T. O’Neill (08:57) “I think the most important thing is that data is a raw ingredient here; it’s a foundation piece for the solution that we’re going to make that’s so good, someone might pay to use it or trade something of value to use it. And as long as that’s intact, I think you’re kind of checking the box as to whether it’s a data product.” — Brian T. O’Neill (12:13)   “I also would say on the data mesh topic, the feeling I got from people who had been to this conference before was that was quite a hyped thing the last couple years. Now, it was not talked about as much, but I think now they’re actually seeing some examples of this working.” — Brian T. O’Neill (16:25)   “My current v2 definition right now is, ‘A data product is a managed, end-to-end software solution that organizes, refines, or transforms data to solve a problem that’s so important customers would pay for it or exchange something of value to use it.’” — Brian T. O’Neill (19:47)   “We know [the product is] of value because someone was willing to pay for it or exchange their time or switch from their old way of doing things to the new way because it has that inherent benefit baked in. That’s really the most important part here that I think any data product manager should fully be aligned with.” — Brian T. O’Neill (21:35)   Links Episode 67 Episode 110 The Definition of Data Product The Data Product Leadership Community Ask me a question (below the recent episodes)

Today I’m answering a question that was submitted to the show by listener Will Angel, who asks how he can prioritize and scale effective discovery throughout the data product development process. Throughout this episode, I explain why discovery work is a process that should be taking place throughout the lifecycle of a project, rather than a defined period at the start of the project. I also emphasize the value of understanding the benefit users will see from the product as the main goal, and how to streamline the effectiveness of the discovery process.  Highlights/ Skip to: Brian introduces today’s topic, Discovery with Data Products, with a listener question (00:28) Why Brian sees discovery work as something that is ongoing throughout the lifecycle of a project (01:53) Brian tackles the first question of how to avoid getting killed by the process overhead of discovery and prioritization (03:38) Brian discusses his take on the question, “What are the ultimate business and user benefits that the beneficiaries hope to get from the product?”(06:02) The value Brian sees in stating anti-goals and anti-personas (07:47) How creative work is valuable despite the discomfort of not being execution-oriented (09:35) Why customer and stakeholder research activities need to be ongoing efforts (11:20) The two modes of design that Brian uses and their distinct purposes (15:09) Brian explains why a clear strategy is critical to proper prioritization (19:36) Why doing a few things really well usually beats out delivering a bunch of features and products that don’t get used (23:24) Brian on why saying “no” can be a gift when used correctly (27:18) How you can join the Data Product Leadership Community for more dialog like this and how to submit your own questions to the show (32:25) Quotes from Today’s Episode “Discovery work, to me is something that largely happens up front at the beginning of a project, but it doesn’t end at the beginning of the project or product initiative, or whatever it is that you’re working on. Instead, I think discovery is a continual thing that’s going on all the time.” — Brian T. O’Neill (01:57) “As tooling gets easier and easier and we need to stand up less infrastructure and basic pipelining in order to get from nothing to something, I think more of the work simply does become the discovery part of the work. And that is always going to feel somewhat inefficient because by definition it is.” — Brian T. O’Neill (04:48) “Measuring [project management metrics] does not tell us whether or not the product is going to be valuable. It just tells us how fast are we writing the code and doing execution against something that may or may not actually have any value to the business at all.” — Brian T. O’Neill (07:33) “How would you measure an improvement in the beneficiaries' lives? Because if you can improve their life in some way—and this often means me at work— the business value is likely to follow there.” — Brian T. O’Neill (18:42) “Without a clear strategy, you’re not going to be able to do prioritization work efficiently because you don’t know what success looks like.” — Brian T. O’Neill (19:49) “Doing a few things really well probably beats delivering a lot of stuff that doesn’t get used. There’s little point in a portfolio of data products that is really wide, but it’s very shallow in terms of value.” — Brian T. O’Neill (23:27) “Anytime you’re going to be changing behavior or major workflows, the non-technical costs and work increase. And we have to figure out, ‘How are we going to market this and evangelize it and make people see the value of it?’ These types of behavior changes are really hard to implement and they need to be figured out during the design of the solution — not afterwards.” — Brian T. O’Neill (26:25) Links designingforanalytics.com/podcast: https://designingforanalytics.com/podcast designingforanalytics.com/community: https://designingforanalytics.com/community