The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Join Arvind Narayanan, a Princeton professor and expert on AI agents and policy, as he unpacks the substance behind AI technology. He discusses the risks of deploying AI agents and the pressing need for better benchmarking to ensure reliability. Delve into his book, which exposes exaggerated AI claims and failed applications. Narayanan also highlights his work on CORE-Bench, aiming to enhance scientific reproducibility and reviews the complex landscape of AI reasoning methods. He wraps up with insights on the tangled web of AI regulation and policy challenges.

Shreya Shankar, a PhD student at UC Berkeley specializing in intelligent data processing, shares her insights on the innovative DocETL system. They discuss how this technology optimizes LLM-powered data pipelines, enhancing analysis of complex documents. Shreya highlights the challenges of data extraction from PDFs, the importance of human feedback in AI systems, and the need for tailored benchmarks in data processing. Real-world applications and the future of agentic systems are also examined, showcasing a visionary path in data management.

Nicholas Carlini, a research scientist at Google DeepMind and winner of the 2024 ICML Best Paper Award, dives into the world of adversarial machine learning. He discusses his groundbreaking work on stealing parts of production language models like ChatGPT. Listeners will learn about the ethical implications of model security, the significance of the embedding layer, and how these advancements raise new security challenges. Carlini also sheds light on differential privacy in AI, questioning its integration with pre-trained models and the future of ethical AI development.

In this discussion, Simon Willison, an independent researcher and creator of Datasette, shares insightful strategies for boosting developer productivity with large language models like ChatGPT and Claude. He reveals how he codes while walking his dog and emphasizes effective prompting and debugging techniques. The conversation dives into the transformative impact of AI on data analysis, the potential of open-source models, and innovative web scraping tools. Listen as he navigates the evolving capabilities and challenges of AI in today's tech landscape!

In this engaging discussion, Shengran Hu, a PhD student at the University of British Columbia, delves into Automated Design of Agentic Systems (ADAS). He shares insights on the spectrum of agentic behaviors and how LLMs can be used for creating novel agent architectures. The conversation highlights the iterative nature of ADAS and its role in revealing emergent behaviors, particularly in complex tasks like the ARC challenge. Shengran also explores practical applications of ADAS in real-world system optimization, emphasizing the balance between innovation and stability.

In this engaging discussion, Peter van der Putten, director of the AI Lab at Pega and an assistant professor at Leiden University, dives deep into the implications of the newly adopted European AI Act. He explains the ethical principles that motivate this regulation and the complexities of applying fairness metrics in real-world AI applications. The conversation highlights the challenges of mitigating bias, the significance of transparency, and how the Act could shape global AI practices, similarly to GDPR's impact on data privacy.

Harrison Chase, co-founder and CEO of LangChain, shares insights from his extensive background in machine learning and MLOps. He discusses the evolution of agentic systems, emphasizing their real-world applications and communication needs. Harrison delves into Retrieval-Augmented Generation (RAG) and the importance of observability tools for enhancing agent development. He also highlights the challenges of transitioning prototypes to production and offers his hot takes on prompting and multi-modal models, providing a glimpse into the future of LLM applications.

Siddhika Nevrekar, Head of AI Hub at Qualcomm Technologies, discusses simplifying on-device AI for developers. She highlights the shift from cloud to local device processing, emphasizing privacy and offline access. The conversation covers challenges in optimizing AI across varied hardware and the collaboration needed between AI frameworks and manufacturers. Siddhika also introduces Qualcomm's AI Hub, aimed at streamlining model testing and fostering innovation in IoT, autonomous vehicles, and enhancing user experiences with AI-integrated solutions.

In this conversation, Ashley Edwards, a member of the technical staff at Runway with past affiliations at Google DeepMind and Uber, reveals the innovative Genie project. They discuss Genie’s ability to create interactive video environments for training reinforcement learning agents without supervision. Topics include the mechanics of latent action models, video tokenization, and dynamics modeling for frame prediction. Ashley highlights the practical implications of Genie and compares it to other models like Sora, mapping out future directions in video generation.

Marius Memmel, a PhD student at the University of Washington, dives into the fascinating world of sim-to-real transfer in robotics. He discusses the complexities of training robots in cluttered environments and how his ASID framework helps improve simulation models. They explore Fisher information's role in optimizing robot learning and the importance of balancing exploration and exploitation. The conversation also highlights his URDFormer model for realistic scene reconstruction, showcasing innovative methods to enhance robotic interactions with their surroundings.