AI Article Readings Beyond The Last Horizon - By Scott Alexander
Apr 24, 2025
Delve into the intricate world of AI forecasting and its limitations regarding time horizons. Discover how humans and AI differ in performance over time, with humans improving while AI tends to plateau. A striking graph reveals AI's success rate in task completion rising to 80%, with exciting acceleration projected between 2024 and 2025. Explore the contrasting planning approaches of AI and humans, while the potential for a transformative coding revolution from AI begs the question: when will superhuman coding capabilities arrive?
AI Snips
Chapters
Transcript
Episode notes
AI Task Length Doubles Rapidly
- AI capability to complete longer coding tasks is doubling roughly every seven months according to Meter.
- This trend suggests rapidly increasing AI task duration capabilities relevant to human equivalence.
AI vs Human Time Horizon Differences
- AI struggles with long-horizon tasks fundamentally differently than humans.
- Unlike humans, giving AI more time beyond its horizon does not improve its task completion performance.
Human Edge in Long Tasks
- Humans have an evolutionary and broad experiential advantage in long-horizon tasks.
- AI training focuses on short, easy-to-grade tasks, limiting their long-horizon task performance currently.