Guests
- David Eason, Principal Product Manager at Trainline
- Billie Bradley, Product Manager, Travel Assistant at Trainline
- Matt Farrelly, Head of AI and Machine Learning at Trainline
Key Takeaways
- AI assistants need both scalable reasoning and deep domain context to be useful.
- Tool design and guardrails are as critical as prompt design in agent systems.
- LLM-as-judge evals make it possible to measure open-ended systems without massive labeling costs.
- Even legacy companies can move fast when they embrace experimentation and tight PM–engineering collaboration.
Chapters:
00:00 Introduction and Team Introductions
00:51 Overview of Trainline's Mission and History
02:30 AI Integration in Trainline's Services
05:08 Challenges and Solutions in AI Implementation
06:52 Building and Iterating the AI Travel Assistant
14:58 User Experience and Guardrails
22:26 Technical Challenges and Solutions
34:29 The Challenge for Product Managers in AI
34:55 Billy's Background in AI
35:42 The Rapid Evolution of AI Technology
37:14 Managing Information Overload
37:58 Collaboration Between Product Managers and Engineers
38:42 Trainline's Approach to Machine Learning
39:36 Scaling Up: From 450 to 700,000 Pages
40:21 Challenges in Data Retrieval and Processing
45:55 Evaluating AI Assistants
48:22 The Role of LLM as Judges
50:19 User Context Simulation for Real-Time Evaluation
01:06:56 Future Directions for Trainline's AI Assistant
