Measuring AI Features and Defining Valuable Interactions

From Alexa to LLMs: Platform experiments, distribution challenges, and how AI is democratizing feature development in travel tech

Tradegate Direct: Europe's most direct online broker – trade for free, efficiently, and directly on the stock exchange. Trade directly here

In this episode, Matthias Keller (CPO @ Kayak) shares strategic lessons about making AI product decisions under uncertainty. With 12 years at Kayak and experience spanning multiple AI waves—from Alexa to LLMs—Matthias offers a masterclass in knowing when to invest early and when to wait.

We dive into:

• Strategic framework for evaluating emerging AI platforms (Alexa, ChatGPT, Vision Pro, Apple Intelligence)
• Why Kayak's 2016 Alexa experiments failed and what that taught them about timing
• The shift from "will this work?" (voice) to "it should work" (LLMs) and why that matters
• Competing with distribution giants: Google Flights has massive advantages, so when do you innovate vs. wait?
• Real talk on platform bets: early ChatGPT plugins, evaluating ROI on new platforms
• How LLMs democratized AI development—engineers now build AI features without data scientists
• User expectation shifts: the "why" questions and what they mean for product strategy
• Commercial complexity: airlines control content, credit cards worth more than tickets
• The "if you build it they may come, if you don't they won't" mindset for CTOs
• Balancing first-mover advantage with execution realities in your org

This conversation offers actionable frameworks for product and engineering leaders navigating AI strategy decisions.

Chapter 1: From Sensors to Travel Tech (00:00 - 09:00) Matthias shares his unconventional path from measuring train wheel vibrations and wireless sensors on mountains to joining Kayak 12 years ago. He discusses the growing complexity of travel booking as airlines unbundled services, creating 15+ ticket options instead of just economy, business, and first class.

Chapter 2: Competitive Landscape & Google Flights (09:00 - 13:00) The challenges of competing in a space where Google Flights has massive distribution advantages. Matthias explains how Kayak maintains its edge through innovation like "hacker fares" (combining one-way tickets from different airlines) and comprehensive comparison across hundreds of partners.

Chapter 3: Platform Evolution - From Alexa to LLMs (13:00 - 28:00) A deep dive into Kayak's platform experiments, from 2016's Alexa skills and Facebook Messenger bots to today's AI mode. Matthias reflects on what worked (flight tracking), what didn't (voice checkout), and the key differences that make LLMs more successful than earlier voice assistants.

Chapter 4: AI Mode & Conversational Search (28:00 - 41:00) How Kayak integrated LLM-powered search across their products, not just as a separate experience. Matthias discusses engagement patterns, the importance of explaining when AI helps versus traditional search, and fascinating user behaviors like asking "why" they should book specific options.

Chapter 5: Distribution in the AI Era (41:00 - 46:00) The "dam breaking" moment when airlines and travel providers might embrace AI platforms for distribution. Discussion of commercial barriers, ecosystem control, and how the industry balance between direct booking and meta-search might shift.

Chapter 6: AI Democratizing Development (46:00 - 50:00) Matthias explains how LLMs enable software engineers to build AI features without data scientists, using examples like Kayak's price check feature that uses vision AI to compare flight prices from screenshots.