The Information Bottleneck EP17: RL with Will Brown
Nov 24, 2025
In this conversation with Will Brown, research lead at Prime Intellect specializing in reinforcement learning (RL) and multi-agent systems, they explore the foundations and practical applications of RL. Will shares insights into the challenges RL faces in LLMs, emphasizing the importance of online sampling and reward models. He discusses multi-agent dynamics, optimization techniques, and the role of game theory in AI development. The discussion also highlights the significance of intermediate results and the future directions for RL in various applications.
AI Snips
Chapters
Transcript
Episode notes
From Theory To Multi‑Agent RL
- Will Brown describes his shift from computer science and philosophy into a PhD focused on algorithmic game theory and multi-agent reinforcement learning.
- He explains studying chaotic multi-agent dynamics and long-horizon learning in many-player settings rather than two-player zero-sum games.
Games As Dynamical Geometry
- Multi‑agent learning can be viewed geometrically as trajectories in a high‑dimensional state space driven by local greedy updates.
- Analyzing when these trajectories are chaotic or tractable guides algorithm design and equilibrium bounds.
Oracle View Bridges Theory And Practice
- Much game theory transfers from discrete action spaces to bounded complexity strategy classes if you assume strong optimization oracles.
- Treating optimization as an oracle clarifies where gradient methods and theoretical guarantees apply.