Current advancements in large language models (LLMs) rely on the refactoring of internet data to capture the intricate thought processes akin to human problem-solving. Internet data, while abundant, does not provide the depth of insight needed for optimal performance; instead, it serves as a valuable resource due to its embedded reasoning traces and knowledge. The true potential for achieving artificial general intelligence (AGI) lies in accessing extensive inner thought trajectories akin to human cognition. By focusing on transforming datasets into formats that reflect this inner monologue, significant progress toward AGI can be realized.
Andrej Karpathy joins Sarah and Elad in this week of No Priors. Andrej, who was a founding team member of OpenAI and former Senior Director of AI at Tesla, needs no introduction. In this episode, Andrej discusses the evolution of self-driving cars, comparing Tesla and Waymo’s approaches, and the technical challenges ahead. They also cover Tesla’s Optimus humanoid robot, the bottlenecks of AI development today, and how AI capabilities could be further integrated with human cognition. Andrej shares more about his new company Eureka Labs and his insights into AI-driven education, peer networks, and what young people should study to prepare for the reality ahead.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Karpathy
Show Notes:
(0:00) Introduction
(0:33) Evolution of self-driving cars
(2:23) The Tesla vs. Waymo approach to self-driving
(6:32) Training Optimus with automotive models
(10:26) Reasoning behind the humanoid form factor
(13:22) Existing challenges in robotics
(16:12) Bottlenecks of AI progress
(20:27) Parallels between human cognition and AI models
(22:12) Merging human cognition with AI capabilities
(27:10) Building high performance small models
(30:33) Andrej’s current work in AI-enabled education
(36:17) How AI-driven education reshapes knowledge networks and status
(41:26) Eureka Labs
(42:25) What young people study to prepare for the future