The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Hierarchical and Continual RL with Doina Precup - #567
Apr 11, 2022
In this engaging conversation, Doina Precup, a Research team lead at DeepMind Montreal and a professor at McGill University, dives into her research on hierarchical and continual reinforcement learning. She discusses how agents can learn abstract representations and the critical role of reward specifications in shaping intelligent behaviors. Doina draws intriguing parallels between hierarchical RL and CNNs while exploring the challenges and future of reinforcement learning in dynamic environments, all while emphasizing the importance of adaptability and multi-level reasoning.
AI Snips
Chapters
Books
Transcript
Episode notes
Reward-Driven Intelligence
- Doina Precup believes reward signals in complex environments can drive agents to develop intuitive intelligence.
- She emphasizes learning abstract representations, especially over time, as crucial for solving complex problems.
Squirrel Intelligence
- Doina Precup uses squirrels' nut-gathering behavior to illustrate reward-driven intelligence development.
- Squirrels develop complex abilities like memory, planning, and deception while maximizing their simple reward function.
Markovian Reward Limitations
- Not all preferences can be translated into Markovian rewards, especially those involving unreachable states or non-Markovian behavior.
- Doina Precup highlights that contradictory preferences in unreachable states can affect reward specification.
