Is There a Hardware for Long Term Memory?

2min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

The first few years of infants is not actually like learning. It's brain maturing. We're born premature. There's a theory along those lines because of the birth canal and the swelling of the brain. And so it's best for the system once it's trained not to remember how it's constructed. I think it's just like the hardware for long term memory is just not fully developed.

Andrej Karpathy is a legendary AI researcher, engineer, and educator. He’s the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford. Please support this podcast by checking out our sponsors:
– Eight Sleep: https://www.eightsleep.com/lex to get special savings
– BetterHelp: https://betterhelp.com/lex to get 10% off
– Fundrise: https://fundrise.com/lex
– Athletic Greens: https://athleticgreens.com/lex to get 1 month of fish oil

EPISODE LINKS:
Andrej’s Twitter: http://twitter.com/karpathy
Andrej’s YouTube: http://youtube.com/c/AndrejKarpathy
Andrej’s Website: http://karpathy.ai
Andrej’s Google Scholar: http://scholar.google.com/citations?user=l8WuQJgAAAAJ
Books mentioned:
The Vital Question: https://amzn.to/3q0vN6q
Life Ascending: https://amzn.to/3wKIsOE
The Selfish Gene: https://amzn.to/3TCo63s
Contact: https://amzn.to/3W3y5Au
The Cell: https://amzn.to/3W5f6pa

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips

SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman

OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(05:41) – Neural networks
(10:45) – Biology
(16:15) – Aliens
(26:27) – Universe
(38:18) – Transformers
(46:34) – Language models
(56:45) – Bots
(1:03:05) – Google’s LaMDA
(1:10:28) – Software 2.0
(1:21:28) – Human annotation
(1:23:25) – Camera vision
(1:28:30) – Tesla’s Data Engine
(1:32:39) – Tesla Vision
(1:39:09) – Elon Musk
(1:44:17) – Autonomous driving
(1:49:11) – Leaving Tesla
(1:54:39) – Tesla’s Optimus
(2:03:45) – ImageNet
(2:06:23) – Data
(2:16:15) – Day in the life
(2:29:31) – Best IDE
(2:36:37) – arXiv
(2:41:06) – Advice for beginners
(2:50:24) – Artificial general intelligence
(3:03:44) – Movies
(3:09:37) – Future of human civilization
(3:13:56) – Book recommendations
(3:20:05) – Advice for young people
(3:21:56) – Future of machine learning
(3:28:44) – Meaning of life