The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Advancing Deep Reinforcement Learning with NetHack, w/ Tim Rocktäschel - #527
Oct 14, 2021
Tim Rocktäschel, a research scientist at Facebook AI Research and associate professor at University College London, dives into the intricate world of training reinforcement learning agents using the complex game NetHack. He discusses the challenges of generalization in simulated environments and the innovative MiniHack framework. The conversation highlights the significance of procedural generation, the intricacies of creating effective scoring systems, and the computational demands for training these advanced AI models. Tim's insights illuminate the future of AI in dynamic settings.
AI Snips
Chapters
Transcript
Episode notes
Generalization in RL
- Tim Rocktäschel's research focuses on training RL agents in simulated environments.
- He aims to improve agent generalization to novel situations, moving beyond constrained environments like Atari.
Overfitting in RL
- Deep learning models, including RL agents, can exploit simplifying assumptions in environments.
- This overfitting hinders generalization to real-world problems, making the real world the only truly unconstrained environment.
NetHack as an RL Environment
- Rocktäschel chose NetHack as an RL environment due to its richness, complexity, and speed.
- NetHack is a procedurally generated dungeon crawler with diverse items and monsters, offering a challenging environment.