SHIFT A List of What to Watch in 2025
Jan 15, 2025
In this engaging discussion, cognitive scientist Gary Marcus shares his predictions for AI in 2025. He explores the contentious topic of artificial general intelligence, critiquing past generative AI predictions. Gary highlights the urgent need for regulation, sharing his experience testifying in the U.S. Senate about the risks of unregulated AI. He balances his critical outlook with an optimistic view of technological advancements, especially in large language models, underscoring the importance of realistic expectations as the field evolves.
AI Snips
Chapters
Books
Transcript
Episode notes
Extrapolation Limits Of Deep Learning
- Gary Marcus argues deep learning systems interpolate well but struggle to extrapolate beyond training data.
- This extrapolation limit explains hallucinations and failures in reasoning and abstract tasks.
Scaling Laws Are Not Forever
- Marcus predicted scaling laws won't hold indefinitely and called for new breakthroughs beyond brute-force data and compute.
- He says recent company leaks and speeches confirm that adding data/compute yields diminishing returns.
Marry Neural Nets With Symbolic AI
- Combine strengths of large language models with classical symbolic systems to improve reliability.
- Pursue neurosymbolic approaches to add deliberative, rule-based reasoning to statistical models.
