MLOps.community

Simba Khadder, founder of Featureform and now at Redis, dives into the fascinating world of context engineering for AI. He argues that context, not models, is the real bottleneck for agents. Simba discusses the evolution of feature stores, emphasizing their ongoing value amidst changing ML economics. He introduces a GraphQL-style semantic layer for better data navigation and details how Redis powers these systems with robust capabilities. Plus, he shares insights on how to improve agent functionality by enhancing context access.

In this engaging discussion, Satish Bhambri, a Senior Data Scientist at Walmart Labs, dives deep into the evolution of machine learning, exploring the transition from RNNs to transformers. He sheds light on the emergence of RAG systems and their ability to ground large language models, tackling issues like hallucinations. Satish explains the benefits of agentic RAG for creating specialized agents and the trade-offs between APIs and agents. He also shares insights on vector database selection and the importance of data freshness in recommendation systems.

Zack Reneau-Wedeen, Head of Product at Sierra, shares insights on revolutionizing AI with context engineering, prioritizing real-world testing over traditional methods. He reveals how AI often feels like a moody coworker and discusses the importance of robust simulations to enhance reliability. Zack advocates for abandoning decision trees in favor of goal-oriented frameworks and explains how Sierra trains graduates to be product-engineering hybrids. He also emphasizes the significance of customer focus to improve AI agents and discusses innovative strategies for scaling and fine-tuning voice interactions.

Spencer Reagan, R&D lead at Airia, specializes in AI-agent orchestration and data governance for regulated environments. He dives into the complexities of agent deployment, discussing how messy data impacts AI performance and why many AI platforms struggle to scale. Reagan offers insights on monitoring agents' actions, enhancing trust through frequent oversight, and emphasizing automation in marketing and HR. He highlights the importance of dynamic rules and identity management for secure agent operations, sharing practical analogies to improve agent design.

In this engaging discussion, expert panelists share insights into the world of e-commerce agents. Arushi Jain, a Microsoft applied scientist, delves into post-training techniques that enhance AI reliability for tasks. Swati Bhatia from Google Cloud talks about using Direct Preference Optimization to fine-tune support routing. Audi Liu from Inworld AI discusses architectural trade-offs in voice models for better accuracy. Isabella Piratininga of iFood highlights personalization challenges in Brazil. Together, they explore the complexities of automating customer interactions and the future of AI.

Ricky Doar, VP of Solutions at Cursor, brings a wealth of experience from leading AI developer tool implementations. He discusses AI as a learnable engineering skill and emphasizes the importance of understanding model capabilities. Ricky warns against over-reliance on AI for strategic decisions and highlights best practices for working with existing codebases. He also shares insights on managing context windows and when to trust AI suggestions, ultimately guiding engineers to make informed decisions while harnessing AI's potential.

Jure Leskovec, a leading AI researcher and Chief Scientist at Kumo.AI, discusses relational foundation models that revolutionize how enterprises harness structured data. He explains the importance of relational data over document-centric AI and proposes raw-data learning to replace feature engineering. Jure highlights using graph neural networks for efficient database representation, the advantages of relational models in recommendations, and successful implementations like DoorDash's 30% accuracy boost. He also emphasizes the cost-effectiveness and efficiency of these models, transforming the landscape of enterprise AI.

Jeff Huber, CEO of Chroma, reveals the challenges of 'context rot,' where AI memory decays, impacting performance. He discusses why traditional benchmarks can mislead developers and explains how Chroma's two-stage retrieval optimizes both recall and precision. The conversation dives into the evolution of search technologies, pitfalls of single embeddings, and the intricacies of personalization in semantic search. Huber emphasizes the need for cleaner, engineered solutions in AI that reduce dependency on fragile systems.

Brooke Hopkins, CEO of Coval and former Waymo lead, sheds light on the evolution of voice AI, highlighting its transition from niche to mainstream. She discusses the vital role of reliability in voice agents, emphasizing strategies like redundancy and latency monitoring to enhance user experience. They delve into practical applications such as customer support and healthcare, while also exploring innovative techniques for context retention and dynamic adjustments during conversations. The conversation also tackles the future of voice autonomy and its realistic timelines.

Rani Radhakrishnan, a Principal at PwC, specializes in AI-managed services and data-driven transformation. She dives into how organizations are shifting from experimentation to realizing ROI with AI solutions. Topics include the need for process standardization, the role of data quality for AI agents, and the importance of human oversight in AI deployment. Rani also contrasts traditional managed services with AI-driven operations, emphasizing continuous optimization and the evolving skill sets required in today's tech landscape.