In this engaging discussion, Richard Sutton, a pioneer in reinforcement learning, and Andrew Barto, an influential figure in the same field, share insights on their groundbreaking work. They delve into the origins of reinforcement learning, discussing its ties to neuroscience and psychology. The duo reflects on their notable contributions, like temporal difference learning, and its applications in AI systems such as AlphaGo. They also explore the future of human-RL relationships, emphasizing the importance of safety and ethical considerations in AI development.
42:39
forum Ask episode
web_stories AI Snips
view_agenda Chapters
auto_awesome Transcript
info_circle Episode notes
insights INSIGHT
Reinforcement Learning As Rediscovered Common Sense
Reinforcement learning formalizes the old, commonsense idea of learning from consequences into a computational framework.
Andrew G. Barto and Richard Sutton framed that rediscovery as making the idea tractable and prominent in AI.
question_answer ANECDOTE
1977 Project That Sparked RL Work
Barto described being hired in 1977 to test the unorthodox idea that neurons act like goal-directed agents learning from consequences.
That project sparked interdisciplinary exploration and led to computational implementations that revived the field.
insights INSIGHT
Temporal-Difference Learning Mirrors Biology
Temporal-difference (TD) learning uses changes in predictions over time as learning signals rather than waiting for final outcomes.
TD later matched dopamine neuron recordings, linking the algorithm to biological reward signals.
Get the Snipd Podcast app to discover more snips from this episode
In this episode of ACM ByteCast, Rashmi Mohan hosts 2024 ACM A.M. Turing Award laureates Andrew Barto and Richard Sutton. They received the Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning, a computational framework that underpins modern AI systems such as AlphaGo and ChatGPT. Barto is Professor Emeritus in the Department of Information and Computer Sciences at the University of Massachusetts, Amherst. His honors include the UMass Neurosciences Lifetime Achievement Award, the IJCAI Award for Research Excellence, and the IEEE Neural Network Society Pioneer Award. He is a Fellow of IEEE and AAAS. Sutton is a Professor in Computing Science at the University of Alberta, a Research Scientist at Keen Technologies (an artificial general intelligence company) and Chief Scientific Advisor of the Alberta Machine Intelligence Institute (Amii). In the past he was a Distinguished Research Scientist at Deep Mind and served as a Principal Technical Staff Member in the AI Department at the AT&T Shannon Laboratory. His honors include the IJCAI Research Excellence Award, a Lifetime Achievement Award from the Canadian Artificial Intelligence Association, and an Outstanding Achievement in Research Award from the University of Massachusetts at Amherst. Sutton is a Fellow of the Royal Society of London, AAAI, and the Royal Society of Canada.
In the interview, Andrew and Richard reflect on their long collaboration together and the personal and intellectual paths that led both researchers into CS and reinforcement learning (RL), a field that was once largely neglected. They touch on interdisciplinary explorations across psychology (animal learning), control theory, operations research, cybernetics, and how these inspired their computational models. They also explain some of their key contributions to RL, such as temporal difference (TD) learning and how their ideas were validated biologically with observations of dopamine neurons. Barto and Sutton trace their early research to later systems such as TD-Gammon, Q-learning, and AlphaGo and consider the broader relationship between humans and reinforcement learning-based AI, and how theoretical explorations have evolved into impactful applications in games, robotics, and beyond.
ACM ByteCast 
