The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Arash Behboodi, Director of Engineering at Qualcomm AI Research, discusses what's on the agenda for this year's NeurIPS conference. He highlights the challenges of differentiable simulation, particularly in wireless systems, and dives into how uncertainty quantification can enhance machine learning models through conformal prediction and entropy. Behboodi also previews innovative demos like on-device video editing and 3D content generation, showcasing Qualcomm's commitment to making cutting-edge AI accessible.

Shirley Wu, Senior Director of Software Engineering at Juniper Networks, leads the discussion on the transformative power of AI and ML in network management. She shares insights on diagnosing cable issues and proactive coverage monitoring, highlighting the benefits of integrating data science. The conversation covers the evolution of roles for network administrators and the innovative Marvis chatbot, which enhances user engagement. Shirley also discusses future directions like proactive testing and the application of smaller ML models to improve efficiency and cost management.

Jason Liu, a freelance AI consultant and creator of the Instructor library, dives deep into retrieval-augmented generation (RAG) systems. He shares key signs of RAG system failures and the tactical steps to diagnose issues. Liu emphasizes building robust test datasets and the importance of data-driven experimentation. He discusses fine-tuning strategies, chunking techniques, and collaboration tools, while also showcasing how future AI models could revolutionize the space. Finally, he highlights his passion for teaching through his AI consulting course.

Today we're joined by Sunil Mallya, CTO and co-founder of Flip AI. We discuss Flip’s incident debugging system for DevOps, which was built using a custom mixture of experts (MoE) large language model (LLM) trained on a novel "CoMELT" observability dataset which combines traditional MELT data—metrics, events, logs, and traces—with code to efficiently identify root failure causes in complex software systems. We discuss the challenges of integrating time-series data with LLMs and their multi-decoder architecture designed for this purpose. Sunil describes their system's agent-based design, focusing on clear roles and boundaries to ensure reliability. We examine their "chaos gym," a reinforcement learning environment used for testing and improving the system's robustness. Finally, we discuss the practical considerations of deploying such a system at scale in diverse environments and much more.The complete show notes for this episode can be found at https://twimlai.com/go/708.

Scott Stephenson, co-founder and CEO of Deepgram, shares his unique journey from particle physics to AI voice technology. He highlights the complexities of building intelligent voice agents, focusing on perception, interaction, and real-time updates. The discussion dives into the transformative potential of AI in customer service, emphasizing federated learning for continuous improvement. Scott also unveils Deepgram's new agent toolkit, showcasing applications across industries like healthcare and food service, and the need for adaptable models in voice interactions.

Tim Rocktäschel, senior staff research scientist at Google DeepMind and AI professor at UCL, explores the tantalizing prospects of artificial superintelligence. He discusses the journey from narrow AI to superhuman capabilities, stressing the necessity of open-ended system development. The conversation also dives into the transformative impact of AI in science and medicine, alongside its role in enhancing debate automation for truth-seeking. With insights from his recent research, he highlights the importance of evolutionary algorithms and addresses challenges like bias in AI.

Lucas García, Principal Product Manager for Deep Learning at MathWorks, dives into the integration of ML in safety-critical systems. He discusses crucial verification and validation processes, highlighting the V-model and its W-adaptation for ML. The conversation shifts to deep learning in aviation, focusing on data quality, model robustness, and interpretability. Lucas also introduces constrained deep learning and convex neural networks, examining the benefits and trade-offs of these approaches while stressing the importance of safety protocols and regulatory frameworks.

Join Arvind Narayanan, a Princeton professor and expert on AI agents and policy, as he unpacks the substance behind AI technology. He discusses the risks of deploying AI agents and the pressing need for better benchmarking to ensure reliability. Delve into his book, which exposes exaggerated AI claims and failed applications. Narayanan also highlights his work on CORE-Bench, aiming to enhance scientific reproducibility and reviews the complex landscape of AI reasoning methods. He wraps up with insights on the tangled web of AI regulation and policy challenges.

Shreya Shankar, a PhD student at UC Berkeley specializing in intelligent data processing, shares her insights on the innovative DocETL system. They discuss how this technology optimizes LLM-powered data pipelines, enhancing analysis of complex documents. Shreya highlights the challenges of data extraction from PDFs, the importance of human feedback in AI systems, and the need for tailored benchmarks in data processing. Real-world applications and the future of agentic systems are also examined, showcasing a visionary path in data management.

Nicholas Carlini, a research scientist at Google DeepMind and winner of the 2024 ICML Best Paper Award, dives into the world of adversarial machine learning. He discusses his groundbreaking work on stealing parts of production language models like ChatGPT. Listeners will learn about the ethical implications of model security, the significance of the embedding layer, and how these advancements raise new security challenges. Carlini also sheds light on differential privacy in AI, questioning its integration with pre-trained models and the future of ethical AI development.