How large of a breakthrough is necessary for dangerous AI?
In order to cause a catastrophe, an AI system would need to be very competent at agentic tasks[1]. The best metric of general agentic capabilities is METR's time horizon. The time horizon measures the length of well-specified software tasks AI systems can do, and is grounded in human baselines, which means AI performance can be closely compared to human performance.
Causing a catastrophe[2] is very difficult. It would likely take many decades, or even centuries, of skilled human labor. Let's use one year of human labor as a lower bound on how difficult it is. This means that AI systems will need to at least have a time horizon of one work-year (2000 hours) in order to cause a catastrophe.
Current AIs have a time horizon of 2 hours, which means it's 1000x lower than the time horizon necessary to cause a catastrophe. This presents a pretty large buffer.
Currently, the time horizon is doubling roughly every half-year. That means that a 1000x increase would take roughly 5 years at the current rate of progress. So, in order for AI to reach a time horizon of 1 work-year within [...]
---
Outline:
(00:11) How large of a breakthrough is necessary for dangerous AI?
(02:04) AI breakthroughs of the recent past
(02:27) Case 1: Transformers
(03:54) Case 2: AlphaFold
(04:30) What is the probability of 1-year time horizons in the next 6 months?
(05:10) Narrowly superhuman AI leading to generally competent AI
(06:27) Would we notice a massive capabilities increase?
(07:44) Conclusion
The original text contained 3 footnotes which were omitted from this narration.
---
First published:
November 11th, 2025
Source:
https://www.lesswrong.com/posts/B5xQwkmWL5wmFNZkX/how-likely-is-dangerous-ai-in-the-short-term
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
