Machine Learning Street Talk (MLST) #045 Microsoft's Platform for Reinforcement Learning (Bonsai)
Feb 28, 2021
Scott Stanfield and Megan Bloemsma from Microsoft's Autonomous Systems team dive into the ambitious Project Bonsai. They discuss its goal to simplify reinforcement learning, making it accessible for developers without PhDs. The conversation highlights the role of machine teaching in enhancing AI training, using real-world applications like balancing robots. They emphasize the need for expert guidance and domain knowledge in overcoming traditional challenges in the field. Innovations in simulation and collaboration are also spotlighted, showcasing a future where complex tasks become manageable.
AI Snips
Chapters
Transcript
Episode notes
Two-Ball Anecdote
- Tim Scarfe uses a two-ball anecdote with Project Moab to illustrate machine learning's limitations.
- A model trained for one task gets confused when given a slightly different input, highlighting generalization issues.
Reinforcement Learning vs. Traditional ML
- Deep reinforcement learning (RL) differs from traditional machine learning by exploring environments without pre-set answers.
- Bonsai aims to optimize and automate physical processes through this exploration.
RL's Challenges
- Alex Erpan's article, "Deep Reinforcement Learning Doesn't Work Yet," highlights RL's challenges, like reward sparsity and generalization.
- Tim Scarfe emphasizes that RL's complexity makes expert consultation crucial, a key aspect of Bonsai's approach.