#416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI
Lex Fridman Podcast
Learning Through Observation Over Language
Large language models accumulate vast knowledge through training on a massive amount of text from the internet. However, the data they are trained on is relatively small compared to the information a four-year-old child absorbs through sensory inputs. Researchers estimate that a child receives about 10 to the 15 bytes of information through the visual cortex in four years of life, significantly more than the 10 to the 13 bytes of training data for language models. This demonstrates that observation and interaction with the real world play a more significant role in learning and knowledge acquisition than language. The insight emphasizes that a considerable portion of what we learn, especially in early development, and even in animals, is not language-based, but rather acquired through sensory experiences and real-world interactions.
