Guests

David Eason, Principal Product Manager at Trainline
Billie Bradley, Product Manager, Travel Assistant at Trainline
Matt Farrelly, Head of AI and Machine Learning at Trainline

Key Takeaways

AI assistants need both scalable reasoning and deep domain context to be useful.
Tool design and guardrails are as critical as prompt design in agent systems.
LLM-as-judge evals make it possible to measure open-ended systems without massive labeling costs.
Even legacy companies can move fast when they embrace experimentation and tight PM–engineering collaboration.

Chapters: 00:00 Introduction and Team Introductions 00:51 Overview of Trainline's Mission and History 02:30 AI Integration in Trainline's Services 05:08 Challenges and Solutions in AI Implementation 06:52 Building and Iterating the AI Travel Assistant 14:58 User Experience and Guardrails 22:26 Technical Challenges and Solutions 34:29 The Challenge for Product Managers in AI 34:55 Billy's Background in AI 35:42 The Rapid Evolution of AI Technology 37:14 Managing Information Overload 37:58 Collaboration Between Product Managers and Engineers 38:42 Trainline's Approach to Machine Learning 39:36 Scaling Up: From 450 to 700,000 Pages 40:21 Challenges in Data Retrieval and Processing 45:55 Evaluating AI Assistants 48:22 The Role of LLM as Judges 50:19 User Context Simulation for Real-Time Evaluation 01:06:56 Future Directions for Trainline's AI Assistant

Tip 4: The 3-2-1 Method [Habit 7: Rest]