Deep Papers The Illusion of Thinking: What the Apple AI Paper Says About LLM Reasoning
10 snips
Jun 20, 2025 The discussion revolves around a compelling new paper from Apple, challenging traditional evaluations of AI reasoning. It reveals how Large Reasoning Models (LRMs) surprisingly falter on complex tasks while Large Language Models (LLMs) shine in simpler scenarios. The conversation dives into the nuances of problem-solving, contrasting human creativity with algorithmic execution, especially with something as intricate as Rubik's cubes. A philosophical debate unfolds, questioning whether the reasoning showcased by AI is truly genuine or merely an illusion.
AI Snips
Chapters
Transcript
Episode notes
Challenging LRM Reasoning
- The paper questions if large reasoning models (LRMs) genuinely reason or just simulate reasoning steps.
- It introduces synthetic puzzles to test reasoning with controlled complexity and logic.
LRMs vs LLMs Token Usage
- LRMs expend more tokens to list reasoning steps and solve puzzles, needing more compute budget.
- LLMs perform well on low complexity puzzles and require fewer tokens but lack detailed reasoning steps.
Reasoning Effort Drops on Hard Tasks
- LRMs show a collapse in reasoning effort as puzzle complexity grows higher.
- Instead of increasing, reasoning tokens used drops sharply when complexity exceeds model capacity.