169 - AI Product Management and UX: What’s New (If Anything) About Making Valuable LLM-Powered Products with Stuart Winter-Tear
May 13, 2025
auto_awesome
Stuart Winter-Tear, Chief Product Officer at Altima's Ventures, shares his extensive expertise in AI product management. He discusses the critical role of user experience in LLM-powered products and argues that many innovations stem from FOMO rather than genuine user needs. Stuart emphasizes the importance of crafting value-driven solutions that prioritize problem-solving over technology. He also explores the complexities of achieving product-market fit and the challenges of building user trust in AI products, highlighting the need for skilled 'translators' between tech and business.
User experience (UX) is essential for AI product success, requiring intuitive interfaces that address complex user needs effectively.
Product managers must balance the unpredictability of AI research with structured engineering processes to ensure timely project delivery.
Successful AI product management demands a focus on real user problems over technologies, driving strategic thinking and sustainable innovation.
Deep dives
The Importance of User Experience in AI Products
User experience (UX) is becoming increasingly critical in the development of AI-powered products, as they need to address complex user needs and expectations effectively. Those who design AI technologies must shift their approach, understanding that users require more than just factual accuracy; they need intuitive, accessible interfaces that enhance usability. The podcast highlights the notion that the evolving landscape of AI products could parallel the introduction of graphical user interfaces, suggesting that future interactions may be more sophisticated than current chat-based interfaces. A strong focus on UX will be essential for companies to foster widespread adoption and generate tangible benefits from their AI solutions.
Challenges of Balancing Research and Engineering Timelines
The podcast discusses the inherent tension between the unpredictability of AI research timelines and the more structured nature of engineering processes. Product managers must navigate this complexity, bridging the gap between researchers and engineers to ensure projects remain on course despite inherent uncertainties. It is emphasized that managing this relationship requires a unique skill set, as product leaders must foster collaboration while being patient with academic researchers who might be hesitant to release less-than-perfect solutions. This balancing act is particularly vital in AI development, where breakthroughs can significantly alter project trajectories.
Navigating the Product Management Landscape in AI
Product management in the AI space requires an understanding of both technological capabilities and market needs, which is crucial for launching successful products. The discussion stresses the importance of focusing on the business context and user problems rather than solely on the technology itself, as many products fail when built as solutions looking for problems. Establishing a clear connection between AI capabilities and practical user needs will drive product success and enable sustained innovation. This approach reinforces the idea that product management should evolve beyond just a focus on delivery to encompass strategic thinking and value generation.
Creating Value with Large Language Models
Large language models (LLMs) are emerging as powerful tools for various industries, allowing businesses to extract insights from vast amounts of unstructured data. The podcast examines how companies across sectors like legal and medical are leveraging LLMs to streamline operations and improve decision-making processes. However, it is crucial to approach LLMs with caution, as their capabilities must be strategically harnessed to avoid potential pitfalls such as misinformation. The combination of LLMs and tailored applications can lead to significant efficiency improvements, thereby increasing ROI and establishing a compelling business case.
The Future of Product Teams in AI Development
As AI technologies evolve, the makeup and functioning of product teams must also adapt to address new challenges and leverage emerging opportunities. The podcast highlights the roles of UX designers, data scientists, and engineers as critical components of successful AI product teams. This multidisciplinary approach fosters innovation and enables teams to create comprehensive solutions that meet user needs while ensuring robust technical foundations. As businesses seek to capitalize on AI advancements, the collaborative synergy within these teams will be instrumental in navigating the complexities of product development.
Today, I'm chatting with Stuart Winter-Tear about AI product management. We're getting into the nitty-gritty of what it takes to build and launch LLM-powered products for the commercial market that actually produce value. Among other things in this rich conversation, Stuart surprised me with the level of importance he believes UX has in making LLM-powered products successful, even for technical audiences.
After spending significant time on the forefront of AI’s breakthroughs, Stuart believes many of the products we’re seeing today are the result of FOMO above all else. He shares a belief that I’ve emphasized time and time again on the podcast–product is about the problem, not the solution. This design philosophy has informed Staurt’s 20-plus year-long career, and it is pivotal to understanding how to best use AI to build products that meet users’ needs.
Highlights/ Skip to
Why Stuart was asked to speak to the House of Lords about AI (2:04)
The LLM-powered products has Stuart been building recently (4:20)
Finding product-market fit with AI products (7:44)
Lessons Stuart has learned over the past two years working with LLM-power products (10:54)
Figuring out how to build user trust in your AI products (14:40)
The differences between being a digital product manager vs. AI product manager (18:13)
Who is best suited for an AI product management role (25:42)
Why Stuart thinks user experience matters greatly with AI products (32:18)
The formula needed to create a business-viable AI product (38:22)
Stuart describes the skills and roles he thinks are essential in an AI product team and who he brings on first (50:53)
Conversations that need to be had with academics and data scientists when building AI-powered products (54:04)
Final thoughts from Stuart and where you can find more from him (58:07)
Quotes from Today’s Episode
“I think that the core dream with GenAI is getting data out of IT hands and back to the business. Finding a way to overlay all this disparate, unstructured data and [translate it] to the human language is revolutionary. We’re finding industries that you would think were more conservative (i.e. medical, legal, etc.) are probably the most interested because of the large volumes of unstructured data they have to deal with. People wouldn’t expect large language models to be used for fact-checking… they’re actually very powerful, especially if you can have your own proprietary data or pipelines. Same with security–although large language models introduce a terrifying amount of security problems, they can also be used in reverse to augment security. There’s a lovely contradiction with this technology that I do enjoy.” - Stuart Winter-Tear (5:58)
“[LLM-powered products] gave me the wow factor, and I think that’s part of what’s caused the problem. If we focus on technology, we build more technology, but if we focus on business and customers, we’re probably going to end up with more business and customers. This is why we end up with so many products that are effectively solutions in search of problems. We’re in this rush and [these products] are [based on] FOMO. We’re leaving behind what we understood about [building] products—as if [an LLM-powered product] is a special piece of technology. It’s not. It’s another piece of technology. [Designers] should look at this technology from the prism of the business and from the prism of the problem. We love to solutionize, but is the problem the problem? What’s the context of the problem? What’s the problem under the problem? Is this problem worth solving, and is GenAI a desirable way to solve it? We’re putting the cart before the horse.” - Stuart Winter-Tear (11:11)
“[LLM-powered products] feel most amazing when you’re not a domain expert in whatever you’re using it for. I’ll give you an example: I’m terrible at coding. When I got my hands on Cursor, I felt like a superhero. It was unbelievable what I could build. Although [LLM products] look most amazing in the hands of non-experts, it’s actually most powerful in the hands of experts who do understand the domain they’re using this technology. Perhaps I want to do a product strategy, so I ask [the product] for some assistance, and it can get me 70% of the way there. [LLM products] are great as a jumping off point… but ultimately [they are] only powerful because I have certain domain expertise.” - Stuart Winter-Tear (13:01)
“We’re so used to the digital paradigm. The deterministic nature of you put in X, you get out Y; it’s the same every time. Probabilistic changes every time. There is a huge difference between what results you might be getting in the lab compared to what happens in the real world. You effectively find yourself building [AI products] live, and in order to do that, you need good communities and good feedback available to you. You need these fast feedback loops. From a pure product management perspective, we used to just have the [engineering] timeline… Now, we have [the data research timeline]. If you’re dealing with cutting-edge products, you’ve got these two timelines that you’re trying to put together, and the data research one is very unpredictable. It’s the nature of research. We don’t necessarily know when we’re going to get to where we want to be.” - Stuart Winter-Tear (22:25)
“I believe that UX will become the #1 priority for large language model products. I firmly believe whoever wins in UX will win in this large language model product world. I’m against fully autonomous agents without human intervention for knowledge work. We need that human in the loop. What was the intent of the user? How do we get that right push back from the large language model to understand even the level of the person that they’re dealing with? These are fundamental UX problems that are going to push UX to the forefront… This is going to be on UX to educate the user, to be able to inject the user in at the right time to be able to make this stuff work. The UX folk who do figure this out are going to create the breakthrough and create the mass adoption.” - Stuart Winter-Tear (33:42)
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
