Experiencing Data w/ Brian T. O’Neill (UX for AI Data Products, SAAS Analytics, Data Product Management)

Natalia Andreyeva, Senior Director of Product Management for the Nexus AI/ML Solution Portfolio at Infor, shares insights on AI product management for supply chain software. She emphasizes the critical role of user experience in B2B data products, advocating for strong customer engagement throughout the development process. The conversation delves into designing for AI, the importance of user discovery, and how to create compelling, trustworthy AI solutions. Natalia also addresses the nuances of integrating emerging technologies and the need for clear communication in user interfaces.

Discover the vital link between user experience and business value in B2B AI products. The discussion emphasizes the importance of qualitative feedback, arguing that relying solely on analytics can obscure crucial user insights. Learn why simply measuring product adoption isn't enough without enhancing users' lives. The focus is on creating solutions that truly resonate with users and foster trust. Insights from UX expert Jared Spool inspire the conversation, aiming to shift product leaders' perspectives on UX.

A challenge I frequently hear about from subscribers to my insights mailing list is how to design B2B data products for multiple user types with differing needs. From dashboards to custom apps and commercial analytics / AI products, data product teams often struggle to create a single solution that meets the diverse needs of technical and business users in B2B settings. If you're encountering this issue, you're not alone!     In this episode, I share my advice for tackling this challenge including the gift of saying "no.” What are the patterns you should be looking out for in your customer research? How can you choose what to focus on with limited resources? What are the design choices you should avoid when trying to build these products? I’m hoping by the end of this episode, you’ll have some strategies to help reduce the size of this challenge—particularly if you lack a dedicated UX team to help you sort through your various user/stakeholder demands.      Highlights/ Skip to  The importance of proper user research and clustering “jobs to be done” around business importance vs. task frequency—ignoring the rest until your solution can show measurable value  (4:29) What “level” of skill to design for, and why “as simple as possible” isn’t what I generally recommend (13:44) When it may be advantageous to use role or feature-based permissions to hide/show/change certain aspects, UI elements, or features  (19:50) Leveraging AI and LLMs in-product to allow learning about the user and progressive disclosure and customization of UIs (26:44) Leveraging the “old” solution of rapid prototyping—which is now faster than ever with AI, and can accelerate learning (capturing user feedback) (31:14) 5 things I do not recommend doing when trying to satisfy multiple user types in your b2b AI or analytics product (34:14)   Quotes from Today’s Episode If you're not talking to your users and stakeholders sufficiently, you're going to have a really tough time building a successful data product for one user – let alone for multiple personas. Listen for repeating patterns in what your users are trying to achieve (tasks they are doing). Focus on the jobs and tasks they do most frequently or the ones that bring the most value to their business. Forget about the rest until you've proven that your solution delivers real value for those core needs. It's more about understanding the problems and needs, not just the solutions. The solutions tend to be easier to design when the problem space is well understood. Users often suggest solutions, but it's our job to focus on the core problem we're trying to solve; simply entering in any inbound requests verbatim into JIRA and then “eating away” at the list is not usually a reliable strategy. (5:52) I generally recommend not going for “easy as possible” at the cost of shallow value. Instead, you’re going to want to design for some “mid-level” ability, understanding that this may make early user experiences with the product more difficult. Why? Oversimplification can mislead because data is complex, problems are multivariate, and data isn't always ideal. There are also “n” number of “not-first” impressions users will have with your product. This also means there is only one “first impression” they have. As such, the idea conceptually is to design an amazing experience for the “n” experiences, but not to the point that users never realize value and give up on the product.  While I'd prefer no friction, technical products sometimes will have to have a little friction up front however, don't use this as an excuse for poor design. This is hard to get right, even when you have design resources, and it’s why UX design matters as thinking this through ends up determining, in part, whether users obtain the promise of value you made to them. (14:21) As an alternative to rigid role and feature-based permissions in B2B data products, you might consider leveraging AI and / or LLMs in your UI as a means of simplifying and customizing the UI to particular users. This approach allows users to potentially interrogate the product about the UI, customize the UI, and even learn over time about the user’s questions (jobs to be done) such that becomes organically customized over time to their needs. This is in contrast to the rigid buckets that role and permission-based customization present. However, as discussed in my previous episode (164 - “The Hidden UX Taxes that AI and LLM Features Impose on B2B Customers Without Your Knowledge”)  designing effective AI features and capabilities can also make things worse due to the probabilistic nature of the responses GenAI produces. As such, this approach may benefit from a UX designer or researcher familiar with designing data products. Understanding what “quality” means to the user, and how to measure it, is especially critical if you’re going to leverage AI and LLMs to make the product UX better. (20:13) The old solution of rapid prototyping is even more valuable now—because it’s possible to prototype even faster. However, prototyping is not just about learning if your solution is on track. Whether you use AI or pencil and paper, prototyping early in the product development process should be framed as a “prop to get users talking.” In other words, it is a prop to facilitate problem and need clarity—not solution clarity. Its purpose is to spark conversation and determine if you're solving the right problem. As you iterate, your need to continually validate the problem should shrink, which will present itself in the form of consistent feedback you hear from end users. This is the point where you know you can focus on the design of the solution. Innovation happens when we learn; so the goal is to increase your learning velocity. (31:35) Have you ever been caught in the trap of prioritizing feature requests based on volume? I get it. It's tempting to give the people what they think they want. For example, imagine ten users clamoring for control over specific parameters in your machine learning forecasting model. You could give them that control, thinking you're solving the problem because, hey, that's what they asked for! But did you stop to ask why they want that control? The reasons behind those requests could be wildly different. By simply handing over the keys to all the model parameters, you might be creating a whole new set of problems. Users now face a "usability tax," trying to figure out which parameters to lock and which to let float. The key takeaway? Focus on addressing the frequency that the same problems are occurring across your users, not just the frequency a given tactic or “solution” method (i.e. “model” or “dashboard” or “feature”) appears in a stakeholder or user request. Remember, problems are often disguised as solutions. We've got to dig deeper and uncover the real needs, not just address the symptoms. (36:19)

Explore the hidden user experience hurdles posed by AI and large language models for B2B customers. Discover how integrating AI with traditional software can enhance value and user satisfaction. Dive into the complex relationship between AI features and user behavior, emphasizing the importance of addressing real user challenges. Learn about the intricacies of auditing AI systems in regulated sectors while maintaining a seamless experience. This discussion offers valuable insights for product leaders seeking to maximize AI's benefits without compromising user trust.

Discover the challenges of quantifying the value of enterprise data products and the vital role of UX teams. Learn how value is subjective and extends beyond numbers. Explore the art of estimation, emphasizing qualitative assessments to communicate impact effectively. Delve into long-term innovation and ethics in measuring success, urging a shift from traditional ROI calculations. Engage with thoughtful insights on audience feedback and community involvement for continuous improvement.

In this discussion, Simon Landry, a Lead UX researcher at Thomson Reuters; Greg Nudelman, a distinguished designer at Sumo Logic; and Paz Perez, a UX designer at Google, explore the intricacies of designing user experiences for AI-driven products. They debate the challenges of ‘AI-first’ thinking, the importance of defining clear problems for LLMs, and the necessity for diverse design teams. The conversation emphasizes that while AI presents opportunities, it can also complicate user interactions, highlighting the critical need for user-centric approaches in AI design.

Discover the fine balance between enhancing customer trust and the puzzles of AI product design. Explore how GenAI and LLMs can bring both benefits and pitfalls to B2B offerings. Learn why merely adding AI features isn't enough; understanding customer needs is key. Unpack the complexities of user experience in AI products, measuring satisfaction while meeting real-world challenges. Dive into the impact of transparency and trust on sales, ensuring the products are not only cutting-edge but truly indispensable to users.

Adam Berke, Chief Product Officer at The Predictive Index, shares his insights on merging company cultures and products after Charma's recent acquisition. He delves into the challenges of integrating two product teams and the vital need for clarity in leadership structures. Berke discusses how behavioral science shapes their hiring practices, enabling data-driven decisions. He also reflects on fostering employee-manager relationships and navigating the complexities of legacy customer expectations while pushing for innovation in a diverse workplace.

Andy Sutton, GM of Data and AI at Endeavour Group, shares his transformation from traditional analytics to a product-led approach in the liquor and hospitality sectors. He discusses the journey to build the 'Spotify for wines' through personalized data strategies that enhance customer experience. Andy highlights the importance of relationships in data-driven environments and the success his team achieved by focusing on user-centric design and agile methodologies. His insights into balancing analytics with real-world user needs illuminate the future of data products.

In this engaging discussion, Anna Jacobson, Operations and Data Partner at Operator Collective, shares her unique journey from construction management to data product design. She opens up about the art of persuading resistant users to embrace data products. Anna emphasizes the importance of user feedback and understanding diverse perspectives for successful product adoption. She also highlights how cultural challenges within organizations can impede progress and how engaging skeptics can turn them into advocates for data insights.