Machine Learning Street Talk (MLST)

#64 Prof. Gary Marcus 3.0

Feb 24, 2022

In this engaging conversation, cognitive scientist Gary Marcus, founder of Robust AI, tackles profound questions about AI and consciousness. He expresses skepticism about AI systems claiming self-awareness and the philosophical debates around consciousness. The discussion dives into the challenges of abstract models and the importance of stable symbolic representations, particularly in self-driving technology. Marcus also reveals insights on extrapolation in high dimensions and scaling laws, emphasizing the complexities of true understanding in neural networks.

51:47

Creator website

Episode guests

Gary Marcus

AI Chapters

Episode notes

Intro

2min

Consciousness and AI: Exploring the Boundaries

23min

Abstract Models and AI Reasoning Challenges

15min

Exploring Extrapolation and Interpolation in High Dimensions

4min

Scaling Laws in AI: Limits and Insights

6min

Exploring Maximum Likelihood and Beyond in Neural Networks

2min

Patreon: https://www.patreon.com/mlst

Discord: https://discord.gg/HNnAwSduud

YT: https://www.youtube.com/watch?v=ZDY2nhkPZxw

We have a chat with Prof. Gary Marcus about everything which is currently top of mind for him, consciousness

[00:00:00] Gary intro

[00:01:25] Slightly conscious

[00:24:59] Abstract, compositional models

[00:32:46] Spline theory of NNs

[00:36:17] Self driving cars / algebraic reasoning

[00:39:43] Extrapolation

[00:44:15] Scaling laws

[00:49:50] Maximum likelihood estimation

References:

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

https://arxiv.org/abs/2201.02177

DEEP DOUBLE DESCENT: WHERE BIGGER MODELS AND MORE DATA HURT

https://arxiv.org/pdf/1912.02292.pdf

Bayesian Deep Learning and a Probabilistic Perspective of Generalization

https://arxiv.org/pdf/2002.08791.pdf

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.