Just Now Possible

Join David Eason, Principal Product Manager at Trainline; Billie Bradley, Product Manager specializing in AI; and Matt Farrelly, Head of Machine Learning Engineering. They dive into the development of Trainline’s AI Travel Assistant and the unique challenges they faced. Discover how combining AI with deep industry knowledge enhances user experience and the importance of well-structured guardrails. They also discuss innovative evaluation methods and the potential of scalable, real-time support for travelers, even in a traditional company.

Noa Reikhav, Head of Product at ZenCity, focuses on integrating community voices into government workflows. Andrew Therriault, VP of Data Science, discusses the importance of accurate sentiment analysis and AI-driven reports. Shota Papiashvili, SVP of R&D, explains their innovative data architecture and workflows. They explore how AI can transform civic engagement, making democracy more responsive. The trio emphasizes the critical role of context and privacy when using AI to represent community insights and the need for user-friendly outputs for non-technical government leaders.

In this chat, Seyna Diop, Chief Product Officer at Neople, and Job Nijenhuis, CTO and co-founder, dive deep into the world of AI digital coworkers. They talk about Neople's innovative approach to building personified AI agents tailored for customer service. Seyna shares fascinating use cases, from handling repetitive tickets to invoicing help. Job discusses the evolution of their technology, moving from simple suggestions to complex automations. They highlight the importance of customer feedback in ensuring quality and building trust with their AI solutions.

Guests: SallyAnn DeLucia, Director of Product, Arize Jack Zhou, Staff Engineer, Arize In this episode, we cover: What tracing, observability, and evals really mean in GenAI applications How Arize used its own platform to build Alyx, its AI agent The role of customer success engineers in surfacing repeatable workflows Why early prototyping looked like messy notebooks and hacked-together local apps How dogfooding shaped Alyx’s evolution and built confidence for launch Why evals start messy, and how Arize layered evals across tool calls, sessions, and system-level decisions The importance of cross-functional, boundary-spanning teams in building AI products What’s next for Alyx: moving from “on rails” workflows to more autonomous, agentic planning loops Resources & Links Arize AI — Sign up for a free account and try Alex Arize Blog — Lessons learned from building AI products Maven AI Evals Course — The course Teresa took to learn about evals (Get 35% off with Teresa’s affiliate link) Cursor — The AI-powered code editor used by the Arize engineering team DataDog — For understanding application traces OpenAI GPT Models — GPT-3.5, GPT-4, and newer models used in early and current versions of Alex Jupyter Notebooks — A tool for combining code, data, and notes, used in Arise’s prototyping Axial Coding Method by Hamel Husain — A framework for analyzing data and designing evals Chapters: 00:00 Introduction to Sally Ann and Jack 01:08 Overview of Arize.ai and Its Core Components 01:44 Deep Dive into Tracing, Observability, and Evals 03:56 Introduction to Alyx: Arize's AI Agent 04:15 The Genesis and Evolution of Alyx 08:51 Challenges and Solutions in Building Alyx 24:33 Prototyping and Early Development of Alyx 26:22 Exploring the Power of Coding Notebooks 26:51 Early Experiments with Alyx 27:59 Challenges with Real Data 29:20 Internal Testing and Dogfooding 31:55 The Importance of Evals 35:16 Developing Custom Evals 43:09 Future Plans for Alyx 47:59 How to Get Started with Alyx

Hamel Husain, a machine learning engineer with over 25 years of experience at GitHub and Airbnb, dives deep into the intricacies of debugging AI products. He shares insights from his work on forecasting Airbnb guest growth, highlighting challenges like data leakage. The conversation uncovers techniques for error analysis in machine learning, the importance of synthetic data, and the pitfalls of AI-generated outputs like hallucinations. Hamel emphasizes the need for systematic improvement and presents practical tips for enhancing AI evaluations.

Thom van der Doef, Principal Product Designer at eSpark, and Ray Lyons, VP of Product & Engineering, dive into the fascinating development of the AI Teacher Assistant. They discuss the evolution from a chatbot interface to a structured workflow tailored for educators. Key insights include how retrieval augmented generation (RAG) and refined metadata are optimizing the assistant's performance, plus lessons learned in semantic versus keyword search. They also explore future enhancements using student data for personalized recommendations, aiming to bridge teacher mandates with real classroom needs.

Ellen Brandenburger, a product leader with expertise in AI and developer communities, shares insights from her journey at Stack Overflow. She discusses how joining the company just before ChatGPT launched led to significant shifts in product focus. Ellen details the formation of Overflow AI and the evolution of conversational search through various iterations, emphasizing the importance of transparency and attribution for developer trust. She also explores Stack's pivot to licensing their extensive Q&A corpus, showcasing how it enhances AI model performance and benchmarks.

How AI products come to life—straight from the builders themselves. In each episode, we dive deep into how teams spotted a customer problem, experimented with AI, prototyped solutions, and shipped real features. We dig into everything from workflows and agents to RAG and evaluation strategies, and explore how their products keep evolving. If you’re building with AI, these are the stories for you. The first full episode drops on Thursday, September 18th. Don't miss it!