The Ticket: Discover the Future of Customer Service, Support, and Experience, with Intercom Introducing Fin Meetups: Conversations with AI Leaders
Dec 3, 2025
Marco Perini, a deployed engineer at LangChain, dives into captivating discussions about the AI landscape. He shares insights on the evolution of Fin, Intercom's AI customer support agent, from its initial hand-coded roots to a sophisticated system powered by LangGraph. The conversation covers optimization strategies, the importance of A/B testing in scaling AI, and the future of voice agents and customer support. Marco also offers valuable advice for aspiring builders of AI agents, emphasizing pragmatism and stakeholder management.
AI Snips
Chapters
Transcript
Episode notes
From Prototype To Large-Scale Product
- Fin began as a GPT-4–based customer support agent that answers queries and can call APIs to take actions.
- It grew from simple hand-coded loops to a complex multi-model production system with trillions of tokens used.
Start Big, Optimize Later
- Start development with the largest, most capable model you can access to validate feasibility before optimizing.
- Optimize later across prompts, retrieval, and custom models to reduce cost and improve performance.
A/B Test Every Production Change
- Build an A/B testing culture and run experiments at scale in production to measure real user outcomes.
- Use hard and soft resolution metrics and hundreds of tests to decide if a change truly improves the product.