The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Join Mido Assran, a research scientist at Meta's FAIR, as he delves into the groundbreaking V-JEPA model, which aims to bridge human and machine intelligence. He explains how V-JEPA's self-supervised training enables efficient learning from unlabeled video data without the distraction of pixel details. Mido also tackles innovations in visual prediction, the use of advanced techniques for video processing, and the complexities of temporal prediction. This insightful conversation highlights the future of AI reasoning beyond generative models.

Sherry Yang, a Senior Research Scientist at Google DeepMind and a PhD candidate at UC Berkeley, discusses her groundbreaking work on video as a universal interface for AI reasoning. She draws parallels between video generation models and language models, highlighting their potential in real-world decision-making tasks. The conversation covers the integration of video in robotics, the challenges of effective labeling, and the exciting applications of interactive simulators. Sherry also unveils UniSim, showcasing the future of engaging with AI-generated environments.

Sayash Kapoor, a Ph.D. student at Princeton University, discusses his research on the societal impact of open foundation models. He highlights the controversies surrounding AI safety and the potential risks of releasing model weights. The conversation delves into critical issues, such as biosecurity concerns linked to language models and the challenges of non-consensual imagery in AI. Kapoor advocates for a unified framework to evaluate these risks, emphasizing the need for transparency and legal protections in AI development.

Akshita Bhagia, a senior research engineer at the Allen Institute for AI, shares her insights on OLMo, an open-source language model that includes a unique dataset and tools for training. She discusses the innovative Dolma dataset, which boasts a three-trillion-token corpus, and Paloma, a benchmarking tool for evaluating model performance. Throughout the conversation, Akshita emphasizes the importance of data transparency, collaborative research, and the challenges faced in training large-scale models, advocating for a shared knowledge approach in AI development.

Ben Prystawski, a PhD student at Stanford blending cognitive science with machine learning, unveils fascinating insights on LLM reasoning. He discusses his recent paper that questions if reasoning exists in LLMs and the effectiveness of chain-of-thought strategies. Delve into how locality in training data fuels reasoning capabilities, and explore the nuances of optimizing prompts for better model performance. The conversation also touches on how our human experiences shape reasoning, enhancing comprehension in artificial intelligence.

Armineh Nourbakhsh, Executive Director at JP Morgan AI Research, dives into the exciting world of DocLLM, a layout-aware large language model designed for document understanding. She shares insights on the evolution of document AI, focusing on multimodal approaches that combine textual and visual data. Nourbakhsh discusses the challenges of training generative models, the intricacies of processing enterprise documents, and strategies to reduce hallucinations in language models, enhancing performance in complex document analysis.

Sanmi Koyejo, an assistant professor at Stanford University, dives into the fascinating world of large language models (LLMs) and their emergent behaviors. He challenges the hype surrounding these models' capabilities, arguing that nonlinear metrics can create illusions of rapid progress. The conversation also discusses his work on trustworthiness in AI, focusing on critical aspects like toxicity and fairness. Sanmi highlights the need for robust evaluation methods as LLMs are integrated into sensitive fields like healthcare and education.

Kamyar Azizzadenesheli, a staff researcher at Nvidia specializing in reinforcement learning, shares exciting insights on the collaboration between RL and large language models. He discusses innovations like ALOHA, a robot learning to fold clothes, and Voyager, an RL agent excelling in Minecraft using GPT-4. The conversation highlights advancements in risk-aware RL, especially in healthcare and finance. Kamyar also predicts how enhanced computational power will shape the future of deep reinforcement learning and facilitate general intelligence.

Ram Sriharsha, VP of Engineering at Pinecone and an expert in large-scale data processing, explores the transformative power of vector databases and retrieval augmented generation (RAG). He discusses the trade-offs between LLMs and vector databases for effective data retrieval. The conversation sheds light on the evolution of RAG applications, the complexities of maintaining fresh enterprise data, and the exciting new features of Pinecone's serverless offering, which enhances scalability and cost efficiency. Ram also shares insights on the future of vector databases in AI.

In this engaging conversation, Ben Zhao, a Neubauer professor of computer science at the University of Chicago, dives into the critical intersection of security and generative AI. He introduces innovative tools like Fawkes, which masks images from facial recognition, and Glaze, designed to protect artists from style mimicry by subtly altering their work. Zhao also unveils Nightshade, a sophisticated defense mechanism that disrupts generative AI's ability to replicate artistic creations, raising vital questions about data poisoning and copyright in the AI era.