TalkRL: The Reinforcement Learning Podcast

Finale Doshi-Velez is a Professor at the Harvard Paulson School of Engineering and Applied Sciences.  This off-the-cuff interview was recorded at UMass Amherst during the workshop day of RL Conference on August 9th 2024.   Host notes: I've been a fan of some of Prof Doshi-Velez' past work on clinical RL and hoped to feature her for some time now, so I jumped at the chance to get a few minutes of her thoughts -- even though you can tell I was not prepared and a bit flustered tbh.  Thanks to Prof Doshi-Velez for taking a moment for this, and I hope to cross paths in future for a more in depth interview. References  Finale Doshi-Velez Homepage @ Harvard  Finale Doshi-Velez on Google Scholar  

In a dynamic discussion, David Silver, a leading professor in reinforcement learning, dives into the nuances of meta-learning and planning algorithms. He explores how function approximators can enhance RL during inference and contrasts human cognition with machine learning systems in tackling complex problems. Silver also discusses the recent advancements in RL algorithms mentioned during his keynote at the RCL 2024, highlighting ongoing innovations in the field.

David Silver, a principal research scientist at DeepMind and a professor at UCL, dives deep into the evolution of reinforcement learning. He discusses the fascinating transition of AlphaFold from RL to supervised learning for protein folding and highlights RL's potential in protein design. Silver also reflects on how personal health impacts research output and shares insights on AlphaZero's learning strategies in various games. He encourages aspiring researchers to embrace boldness in their endeavors and sketches his journey towards advancing artificial general intelligence.

Vincent Moens, Applied ML Research Scientist at Meta and author of TorchRL, discusses the design philosophy and challenges in creating a versatile reinforcement learning library. He also shares his research journey from medicine to ML, evolution of RL perceptions in the AI community, and encourages active engagement in the open-source community.

Arash Ahmadian discusses preference training in language models, exploring methods like PPO. The podcast dives into reinforced leave one out method, reinforced vs vanilla policy gradient in deep RL, and token-level actions. Reward structures and optimization techniques in RLHF are also explored, emphasizing the importance of curated reward signals.

Glen Berseth is an assistant professor at the Université de Montréal, a core academic member of the Mila - Quebec AI Institute, a Canada CIFAR AI chair, member l'Institute Courtios, and co-director of the Robotics and Embodied AI Lab (REAL).  Featured Links  Reinforcement Learning Conference  Closing the Gap between TD Learning and Supervised Learning--A Generalisation Point of View Raj Ghugare, Matthieu Geist, Glen Berseth, Benjamin Eysenbach

A Research scientist at OpenAI discusses information theory and RL, joint predictions, and Epistemic Neural Networks. They explore challenges in reinforcement learning, handling uncertainty, and balancing exploration vs exploitation. The podcast delves into the importance of joint predictive distributions, Thompson sampling approximation, and uncertainty frameworks in Large Language Models (LLMs).

Sharath Chandra Raparthy, an AI Resident at FAIR at Meta, discusses in-context learning for sequential decision tasks, training models to adapt to unseen tasks and randomized environments, properties of data for in-context learning, burstiness and trajectories in transformers, and the use of G flow nets in sampling from complex distributions.

Pierluca D'Oro and Martin Klissarov discuss their recent work on 'Motif, Intrinsic Motivation from AI Feedback' and its application in NetHack. They also explore the similarities between RL and Learning from Preferences, the challenges of training an RL agent for NetHack, the gap between RL and language models, and the difference between return and loss landscapes in RL.

Martin Riedmiller, a research scientist and team lead at DeepMind, discusses using reinforcement learning to control the magnetic field in a fusion reactor. They explore challenges in the TOCOMAC project, reward design, designing actor and critic networks, DQN and NFQ algorithms, the importance of explainability in RL systems, and the horde architecture for collecting experience.