In this episode, Kamyar Azizzadenesheli, a staff researcher at Nvidia, updates us on the latest developments in reinforcement learning (RL) and how large language models (LLMs) are pushing RL performance forward. He shares insights on applications of LLMs in robotics, such as a robot that can learn to fold clothes, and an RL agent that outperforms prior systems at playing Minecraft. The risks of RL-based decision-making in finance, healthcare, and agriculture are also discussed, along with predictions for the future of deep reinforcement learning.

Ram Sriharsha, VP of engineering at Pinecone, discusses the advantages and complexities of retrieval augmented generation (RAG) with vector databases. He talks about building and deploying real-world RAG-based applications, as well as Pinecone's new serverless offering that enables on-demand data loading, flexible scaling, and cost-effective query processing. Ram shares his perspective on the future of vector databases in helping enterprises deliver RAG systems.

Ben Zhao, a Neubauer professor of computer science at the University of Chicago, discusses his research on security and generative AI. They explore Fawkes, a tool that cloaks images to shield individuals from facial recognition models. They also talk about Glaze, a defense against style mimicry for artists, and Nightshade, a tool for artists to break generative AI models trained on their images.

In this episode, Dan Friedman, a PhD student in the Princeton NLP group, shares his research on mechanistic interpretability for transformer models. They discuss the limitations of prior approaches, the challenges of designing for interpretability, and the process of training and decompiling transformer models. They also explore the constraints in implementing an objective function and the use of probabilistic programs in the learning transformer framework. Additionally, they cover finding a representation and searching within a constrained space using probability distribution and variational inference.

Thomas Dietterich, distinguished professor emeritus at Oregon State University, discusses large language models (LLMs) and their limitations, including lack of modularity, hallucinations, and struggles with novel situations. The conversation also explores uncertainty quantification in machine learning models, the concept of hallucination in LLMs, emergent properties in machine learning, promising opportunities in AI research, and challenges in mixing instructions and parameters in language models.

Naila Murray, director of AI research at Meta, discusses the latest trends in computer vision, including controllable generation, visual programming, 3D Gaussian splatting, and multimodal models. She shares insights on open source projects like Segment Anything, ControlNet, and DINOv2. Naila also talks about exciting opportunities in the field and predicts future advancements in computer vision.

Ed Anuff, chief product officer at DataStax, discusses vector databases, embedding models, and the future of AI infrastructure. They explore the underpinnings of vector databases and their role in serving up relevant results for AI assistants. They also touch on the challenges of maintaining relevancy and scalability, using live data in AI conversational experiences, and the intersection of vector databases and AI in data engineering and software architecture.

Markus Nagel, research scientist at Qualcomm AI Research, discusses his accepted papers at NeurIPS 2023. Topics include tackling activation quantization issues, comparing pruning and quantization, using scalarization in multi-domain learning, applying geometric algebra with equivariance to transformers, and deductive verification of chain of thought reasoning.

Michael Kearns, Professor at the University of Pennsylvania, discusses the challenges of responsible AI in the generative era. Topics include service card metrics, privacy, hallucinations, RLHF, LLM evaluation benchmarks, Clean Rooms ML, and secure data handling in machine learning.

Mike Miller, director of product at AWS responsible for the company’s "edutainment" products, discusses AWS PartyRock, a no-code generative AI app builder. They also explore previous tools including DeepLens, DeepRacer, and DeepComposer. The podcast covers the functionality and models within PartyRock, as well as the importance of hands-on learning and the playful nature of AI apps.