ToKCast Ep 255: Does this research explain how LLMs work?
43 snips
Jan 14, 2026 
Vishal Misra, a computer scientist known for his work on the 'Bayesian Attention Trilogy', joins to demystify language models. They discuss how LLMs work not through creativity but by mapping human explanations without true understanding. Misra argues these models, bound by their training data, lack the ability to innovate concepts or create new scientific knowledge. The conversation also touches on the limitations of Bayesian reasoning and the need for new architectures to achieve artificial general intelligence.
AI Snips
Chapters
Books
Transcript
Episode notes
Transformers Had An Emergent Mystery
- The transformer breakthrough created powerful emergent capabilities not obvious from low-level descriptions.
- Brett Hall emphasizes we lacked a clear high-level explanation until recent Bayesian-attention work.
Bayes Compares, It Does Not Create
- Bayesian reasoning relies on known priors and cannot invent brand-new explanations.
- Brett Hall uses the malaria example to show Bayes helps compare existing hypotheses, not conjure new ones.
Science Is Conjecture, Not Induction
- Scientific progress requires conjecture and refutation rather than mere induction.
- Brett Hall highlights Popperian problem-solving as essential for creating new explanations.
