"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis The Customer Service Revolution: Building Fin, with Eoghan McCabe & Fergal Reid of Intercom
105 snips
Oct 5, 2025 
Eoghan McCabe, CEO of Intercom, and Fergal Reid, Chief AI Officer, dive into the creation of their AI customer service agent, FIN. They reveal how FIN achieved a remarkable 65% resolution rate through custom model optimization rather than off-the-shelf solutions. The duo discusses their unique $0.99 per resolution pricing model, aligning customer value with cost. They also tackle the challenges of automation, balancing speed with trust, and the ongoing need for intelligent design in customer service.
AI Snips
Chapters
Books
Transcript
Episode notes
Context Engineering Drives Gains
- Intercom found that model improvements contributed only a few percentage points to FIN's gains.
- Most of the 30+ point increase came from optimizing retrieval, re-ranking, prompting, and workflows.
Make Monitoring A Team Sport
- Distribute keeping-up responsibilities across a core AI group rather than one gatekeeper.
- Create space for experimentation so new models get quickly validated and integrated.
Production A/B Tests Trump Evals
- Offline evals are helpful but insufficient for real user behavior.
- Always validate with large-scale A/B tests in production to detect tiny but meaningful changes.
