Mixture of Experts Why language models hallucinate, revisiting Amodei’s code prediction and AI in the job market
13 snips
Sep 12, 2025 Ailey McConnan, a tech news writer at IBM Think, shares the week's AI headlines, while Chris Hay, a distinguished engineer, dives deep into the intricacies of language model hallucinations and their implications for reliability. Skyler Speakman, a senior research scientist, discusses the evolving role of AI in coding jobs and the significant impact on the job market. They also explore the fascinating potential of running language models on ultra-compact hardware, reshaping how we think about AI technology in our everyday lives.
AI Snips
Chapters
Transcript
Episode notes
Hallucinations Are Incentive Problems
- OpenAI's paper reframes hallucination as an incentive and training issue rather than a pure modeling limitation.
- Reward functions and evals push models to guess because guessing can return higher scores than saying "I don't know."
RL And Evals Inflate Guessing Behavior
- Reinforcement learning and binary evals have amplified guessing by rewarding right-or-wrong outcomes with no partial credit.
- Model providers optimize benchmark scores, which discourages models from saying "I don't know."
Accuracy Isn’t The Same As Groundedness
- Accuracy and hallucination are distinct metrics; improving accuracy doesn't necessarily reduce hallucinations.
- Some statements are ungroundable from context so calibration alone cannot eliminate false assertions.