#416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI
Lex Fridman Podcast
Training LLMs and Auto-Regressive Prediction
LLMs are trained by masking words in a text and training a neural net to predict the missing words. The neural net predicts the next word by generating a probability distribution over all possible words in the dictionary and sampling a word based on this distribution. This process is called auto-regressive prediction, which is why these LLMs are referred to as auto-regressive LLMs.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
