Kyle Nesbit, a longtime Googler with 17 years of experience in AI and distributed systems, shares invaluable insights about building AI at scale. He discusses the challenges of transitioning traditional engineering teams to embrace LLM development and underscores the importance of starting with solid evaluation metrics. Kyle reveals strategies for iterative improvement, tackling data discovery issues, and balancing product quality during scaling. He also provides a peek into the real story behind AI demos and the complexities of integration within organizations.