MLOps.community

Zack Reneau-Wedeen, Head of Product at Sierra, shares insights on revolutionizing AI with context engineering, prioritizing real-world testing over traditional methods. He reveals how AI often feels like a moody coworker and discusses the importance of robust simulations to enhance reliability. Zack advocates for abandoning decision trees in favor of goal-oriented frameworks and explains how Sierra trains graduates to be product-engineering hybrids. He also emphasizes the significance of customer focus to improve AI agents and discusses innovative strategies for scaling and fine-tuning voice interactions.

Spencer Reagan, R&D lead at Airia, specializes in AI-agent orchestration and data governance for regulated environments. He dives into the complexities of agent deployment, discussing how messy data impacts AI performance and why many AI platforms struggle to scale. Reagan offers insights on monitoring agents' actions, enhancing trust through frequent oversight, and emphasizing automation in marketing and HR. He highlights the importance of dynamic rules and identity management for secure agent operations, sharing practical analogies to improve agent design.

In this engaging discussion, expert panelists share insights into the world of e-commerce agents. Arushi Jain, a Microsoft applied scientist, delves into post-training techniques that enhance AI reliability for tasks. Swati Bhatia from Google Cloud talks about using Direct Preference Optimization to fine-tune support routing. Audi Liu from Inworld AI discusses architectural trade-offs in voice models for better accuracy. Isabella Piratininga of iFood highlights personalization challenges in Brazil. Together, they explore the complexities of automating customer interactions and the future of AI.

Ricky Doar, VP of Solutions at Cursor, brings a wealth of experience from leading AI developer tool implementations. He discusses AI as a learnable engineering skill and emphasizes the importance of understanding model capabilities. Ricky warns against over-reliance on AI for strategic decisions and highlights best practices for working with existing codebases. He also shares insights on managing context windows and when to trust AI suggestions, ultimately guiding engineers to make informed decisions while harnessing AI's potential.

Jure Leskovec, a leading AI researcher and Chief Scientist at Kumo.AI, discusses relational foundation models that revolutionize how enterprises harness structured data. He explains the importance of relational data over document-centric AI and proposes raw-data learning to replace feature engineering. Jure highlights using graph neural networks for efficient database representation, the advantages of relational models in recommendations, and successful implementations like DoorDash's 30% accuracy boost. He also emphasizes the cost-effectiveness and efficiency of these models, transforming the landscape of enterprise AI.

Jeff Huber, CEO of Chroma, reveals the challenges of 'context rot,' where AI memory decays, impacting performance. He discusses why traditional benchmarks can mislead developers and explains how Chroma's two-stage retrieval optimizes both recall and precision. The conversation dives into the evolution of search technologies, pitfalls of single embeddings, and the intricacies of personalization in semantic search. Huber emphasizes the need for cleaner, engineered solutions in AI that reduce dependency on fragile systems.

Brooke Hopkins, CEO of Coval and former Waymo lead, sheds light on the evolution of voice AI, highlighting its transition from niche to mainstream. She discusses the vital role of reliability in voice agents, emphasizing strategies like redundancy and latency monitoring to enhance user experience. They delve into practical applications such as customer support and healthcare, while also exploring innovative techniques for context retention and dynamic adjustments during conversations. The conversation also tackles the future of voice autonomy and its realistic timelines.

Rani Radhakrishnan, a Principal at PwC, specializes in AI-managed services and data-driven transformation. She dives into how organizations are shifting from experimentation to realizing ROI with AI solutions. Topics include the need for process standardization, the role of data quality for AI agents, and the importance of human oversight in AI deployment. Rani also contrasts traditional managed services with AI-driven operations, emphasizing continuous optimization and the evolving skill sets required in today's tech landscape.

In this engaging discussion, Andy Pernsteiner, Field CTO at VAST Data, dives into the complexities of building robust AI infrastructures. He highlights the critical gap between prototypes and production systems, emphasizing the importance of unified data and real-time processing. Andy reveals how GPU downtime can escalate costs dramatically and advocates for chaos engineering to ensure reliability. He also shares insights on workflow automation, the need for empathy between tech teams, and the advantages of separating logic from data for scalability. This conversation is a must-listen for anyone in the AI space!

Jeff Schwartzentruber, a Senior Machine Learning Scientist at eSentire, dives into the evolving landscape of AI in cybersecurity. He reveals the shift from signature-based detection to dynamic anomaly detection, tackling issues like alert fatigue in Security Operations Centers. The conversation explores the risks associated with AI agents, including prompt injections and the need for visibility. Jeff highlights how defenders and attackers use Generative AI, emphasizing the importance of maintaining organizational truth amid rising deception risks.