Machine Learning Street Talk (MLST) #49 - Meta-Gradients in RL - Dr. Tom Zahavy (DeepMind)
12 snips
Mar 23, 2021 In this conversation, Dr. Tom Zahavy, a Research Scientist at DeepMind specializing in reinforcement learning, discusses his journey into AI and the potential of reinforcement learning for achieving artificial general intelligence. Alongside Robert Lange, a PhD candidate and insightful blogger, they delve into the concept of meta-gradients, exploring their role in optimizing learning dynamics and hyperparameter tuning. The duo also tackles the challenges of balancing exploration and exploitation, and the significance of recognizing patterns in developing intelligent systems.
AI Snips
Chapters
Transcript
Episode notes
Meta-Gradients in Single Environments
- Meta-gradients adapt algorithms to specific environments, improving learning within that environment.
- This differs from general algorithms designed for any environment, which sacrifice peak performance in a single environment.
Single vs. Multiple Learning Runs
- Running one long learning run may not be enough because we lack a perfect world model.
- Testing on various environments enhances understanding of an algorithm's generality.
Transferability in Meta-Learning
- Transferring learned hyperparameter schedules is harder in single-lifetime meta-learning due to overfitting.
- Black box meta-learning allows transferring discovered objectives to new tasks.