MLOps.community

Jure Leskovec, a leading AI researcher and Chief Scientist at Kumo.AI, discusses relational foundation models that revolutionize how enterprises harness structured data. He explains the importance of relational data over document-centric AI and proposes raw-data learning to replace feature engineering. Jure highlights using graph neural networks for efficient database representation, the advantages of relational models in recommendations, and successful implementations like DoorDash's 30% accuracy boost. He also emphasizes the cost-effectiveness and efficiency of these models, transforming the landscape of enterprise AI.

Jeff Huber, CEO of Chroma, reveals the challenges of 'context rot,' where AI memory decays, impacting performance. He discusses why traditional benchmarks can mislead developers and explains how Chroma's two-stage retrieval optimizes both recall and precision. The conversation dives into the evolution of search technologies, pitfalls of single embeddings, and the intricacies of personalization in semantic search. Huber emphasizes the need for cleaner, engineered solutions in AI that reduce dependency on fragile systems.

Brooke Hopkins, CEO of Coval and former Waymo lead, sheds light on the evolution of voice AI, highlighting its transition from niche to mainstream. She discusses the vital role of reliability in voice agents, emphasizing strategies like redundancy and latency monitoring to enhance user experience. They delve into practical applications such as customer support and healthcare, while also exploring innovative techniques for context retention and dynamic adjustments during conversations. The conversation also tackles the future of voice autonomy and its realistic timelines.

Rani Radhakrishnan, a Principal at PwC, specializes in AI-managed services and data-driven transformation. She dives into how organizations are shifting from experimentation to realizing ROI with AI solutions. Topics include the need for process standardization, the role of data quality for AI agents, and the importance of human oversight in AI deployment. Rani also contrasts traditional managed services with AI-driven operations, emphasizing continuous optimization and the evolving skill sets required in today's tech landscape.

In this engaging discussion, Andy Pernsteiner, Field CTO at VAST Data, dives into the complexities of building robust AI infrastructures. He highlights the critical gap between prototypes and production systems, emphasizing the importance of unified data and real-time processing. Andy reveals how GPU downtime can escalate costs dramatically and advocates for chaos engineering to ensure reliability. He also shares insights on workflow automation, the need for empathy between tech teams, and the advantages of separating logic from data for scalability. This conversation is a must-listen for anyone in the AI space!

Jeff Schwartzentruber, a Senior Machine Learning Scientist at eSentire, dives into the evolving landscape of AI in cybersecurity. He reveals the shift from signature-based detection to dynamic anomaly detection, tackling issues like alert fatigue in Security Operations Centers. The conversation explores the risks associated with AI agents, including prompt injections and the need for visibility. Jeff highlights how defenders and attackers use Generative AI, emphasizing the importance of maintaining organizational truth amid rising deception risks.

Jaipal Singh Goud, a Solutions Architect at Prem AI, dives into the exciting world of fine-tuning small language models for personalized AI agents. He discusses the contrast between general LLMs and company-specific models, addressing privacy and data control concerns. Jaipal also explores the complementary roles of fine-tuning and RAG systems in query improvement. He emphasizes the importance of user observation for fine-tuning decision-making patterns and envisions a future with countless personalized models, dynamically chosen for each task.

David Jayatillake, an experienced AI leader and former VP at Cube.dev, delves into the intricacies of semantic layers and their crucial role in data management. He critiques proprietary BI tools for locking companies into confusing ecosystems, advocating for open-source solutions. The discussion extends to how AI agents can streamline data workflows by automating repetitive tasks and enhancing queryability. Jayatillake also highlights the potential of LLMs in building semantic layers and the significance of company-specific definitions for effective data analysis.

Siddharth Bidasaria, a key member of the Claude Code team at Anthropic, shares insights into the innovative coding product's journey. He reveals how Claude Code evolved from a terminal prototype, attracting immediate internal interest. The conversation highlights user-driven improvements like local file tools that enhanced workflow, and the importance of test-driven development for reliable AI code. Siddharth also discusses the balance between model steerability and user friction, plus exciting future possibilities with sub-agents and customizable permissions.

Biswaroop Bhattacharjee, a Senior ML Engineer at Prem AI, dives into the fascinating world of AI memory systems. He discusses Cortex, an innovative framework inspired by human cognition, highlighting how it manages long-term and multimodal memories. The conversation challenges the boundaries of agentic memory, weighing the necessity of forgetting and the implications of memory consolidation. Biswaroop also shares insights into hierarchical collections, retrieval techniques, and the pursuit of integrating vision and audio for a richer AI memory experience.