Just Now Possible

Steven Payne, Product Manager at Perk, Gabriel Stock, Senior Engineering Manager, and Philipe Steiff, Senior Software Engineer, dive into the world of voice AI. They discuss how they automated over 10,000 weekly calls to ensure hotels charge virtual cards, overcoming challenges with prompt engineering and text-to-speech. The team shares their surprising findings from hotel interactions, refining conversational stages, and the importance of manual call reviews despite advanced metrics. Their innovative journey uncovered operational issues, paving the way for future AI applications.

Martin Siniawski, CEO of Rest, and Ignacio, CTO, dive into the creation of an AI sleep coach. They discuss how podcast listener behavior illuminated the sleep challenge. The duo explains their choice of voice interaction for warmth and trust in their product, built on clinically proven CBT-I principles. They reveal innovative features like memory systems for personalized user experiences and dynamic agendas for daily check-ins. Plus, they share insights on ethical boundaries in health tech and the importance of expert feedback for iterative improvements.

Claire Smid, an AI engineer at Xelix, Emilija Gransaull, back-end tech lead, and Talal A., product manager, dive deep into revolutionizing vendor communications through AI-driven help desks. They share insights on automating high-impact tasks like invoice status inquiries, utilizing an effective email processing pipeline, and creating user-friendly interfaces for seamless adoption. The team highlights the importance of measuring match confidence and response accuracy, while discussing their innovative Carpaccio development process for rapid feedback.

Join Lawrence Jones and Ed Dean from Incident.io as they dive into the revolutionary impact of AI on incident response. Lawrence, a founding engineer leading the AI team, and Ed, the product lead for AI, share insights on how AI compresses investigation times from hours to mere minutes. They discuss the importance of user trust in AI, simplifying complex systems, and using past incidents to improve future responses. Explore their innovative Slack integration and the fascinating concept of AI as a company's immune system in action!

Join David Eason, Principal Product Manager at Trainline; Billie Bradley, Product Manager specializing in AI; and Matt Farrelly, Head of Machine Learning Engineering. They dive into the development of Trainline’s AI Travel Assistant and the unique challenges they faced. Discover how combining AI with deep industry knowledge enhances user experience and the importance of well-structured guardrails. They also discuss innovative evaluation methods and the potential of scalable, real-time support for travelers, even in a traditional company.

Noa Reikhav, Head of Product at ZenCity, focuses on integrating community voices into government workflows. Andrew Therriault, VP of Data Science, discusses the importance of accurate sentiment analysis and AI-driven reports. Shota Papiashvili, SVP of R&D, explains their innovative data architecture and workflows. They explore how AI can transform civic engagement, making democracy more responsive. The trio emphasizes the critical role of context and privacy when using AI to represent community insights and the need for user-friendly outputs for non-technical government leaders.

In this chat, Seyna Diop, Chief Product Officer at Neople, and Job Nijenhuis, CTO and co-founder, dive deep into the world of AI digital coworkers. They talk about Neople's innovative approach to building personified AI agents tailored for customer service. Seyna shares fascinating use cases, from handling repetitive tickets to invoicing help. Job discusses the evolution of their technology, moving from simple suggestions to complex automations. They highlight the importance of customer feedback in ensuring quality and building trust with their AI solutions.

Guests: SallyAnn DeLucia, Director of Product, Arize Jack Zhou, Staff Engineer, Arize In this episode, we cover: What tracing, observability, and evals really mean in GenAI applications How Arize used its own platform to build Alyx, its AI agent The role of customer success engineers in surfacing repeatable workflows Why early prototyping looked like messy notebooks and hacked-together local apps How dogfooding shaped Alyx’s evolution and built confidence for launch Why evals start messy, and how Arize layered evals across tool calls, sessions, and system-level decisions The importance of cross-functional, boundary-spanning teams in building AI products What’s next for Alyx: moving from “on rails” workflows to more autonomous, agentic planning loops Resources & Links Arize AI — Sign up for a free account and try Alex Arize Blog — Lessons learned from building AI products Maven AI Evals Course — The course Teresa took to learn about evals (Get 35% off with Teresa’s affiliate link) Cursor — The AI-powered code editor used by the Arize engineering team DataDog — For understanding application traces OpenAI GPT Models — GPT-3.5, GPT-4, and newer models used in early and current versions of Alex Jupyter Notebooks — A tool for combining code, data, and notes, used in Arise’s prototyping Axial Coding Method by Hamel Husain — A framework for analyzing data and designing evals Chapters: 00:00 Introduction to Sally Ann and Jack 01:08 Overview of Arize.ai and Its Core Components 01:44 Deep Dive into Tracing, Observability, and Evals 03:56 Introduction to Alyx: Arize's AI Agent 04:15 The Genesis and Evolution of Alyx 08:51 Challenges and Solutions in Building Alyx 24:33 Prototyping and Early Development of Alyx 26:22 Exploring the Power of Coding Notebooks 26:51 Early Experiments with Alyx 27:59 Challenges with Real Data 29:20 Internal Testing and Dogfooding 31:55 The Importance of Evals 35:16 Developing Custom Evals 43:09 Future Plans for Alyx 47:59 How to Get Started with Alyx

Hamel Husain, a machine learning engineer with over 25 years of experience at GitHub and Airbnb, dives deep into the intricacies of debugging AI products. He shares insights from his work on forecasting Airbnb guest growth, highlighting challenges like data leakage. The conversation uncovers techniques for error analysis in machine learning, the importance of synthetic data, and the pitfalls of AI-generated outputs like hallucinations. Hamel emphasizes the need for systematic improvement and presents practical tips for enhancing AI evaluations.

Thom van der Doef, Principal Product Designer at eSpark, and Ray Lyons, VP of Product & Engineering, dive into the fascinating development of the AI Teacher Assistant. They discuss the evolution from a chatbot interface to a structured workflow tailored for educators. Key insights include how retrieval augmented generation (RAG) and refined metadata are optimizing the assistant's performance, plus lessons learned in semantic versus keyword search. They also explore future enhancements using student data for personalized recommendations, aiming to bridge teacher mandates with real classroom needs.