Few of the world’s great scientists have given more thought to the existential threats to humanity than the irrepressible British cosmologist and astronomer Martin Rees. He’s the co-founder of Cambridge University’s Centre for Existential Risk as well as the author of the 2003 book Our Final Hour. So it’s striking that Rees has a quite different take on the existential risk of artificial intelligence technology than many AI doomers including yesterday’s guest, the 2024 Physics Nobel laureate Geoffrey Hinton. For Rees, bio-threats and network collapse represents the most dangerous technological threats to humanity in the near future. Unlike nuclear weapons, which require massive detectable infrastructure, Rees warns, dangerous pathogens can be engineered in small, unmonitored laboratories. Meanwhile, our civilization's complete dependence on interconnected global networks means system failures could trigger catastrophic societal breakdown within days. Apocalypse now? Perhaps. But, according to the prescient Rees, we are preparing for the wrong apocalypse.

1. AI's Real Danger Isn't Superintelligence—It's System Dependency

Rees is "very skeptical" about AI takeover scenarios. Instead, he worries about our over-dependence on globe-spanning networks that control electricity grids and internet infrastructure. When these fail—whether from cyberattacks or malfunctions—society could collapse within "two or three days."

2. Bio-Threats Are Uniquely Undetectable and Unstoppable

Unlike nuclear weapons that require massive, monitorable facilities, dangerous pathogens can be engineered in small, undetected laboratories. "Gain of function" experiments could create bioweapons far worse than COVID, and preventing this would require impossible levels of surveillance over anyone with relevant expertise.

3. We're Living Through a Uniquely Dangerous Era

Rees believes "the prospect of a catastrophe in the next 10 or 20 years is perhaps higher than it's ever been." We're the first species in Earth's history capable of changing the entire planet—for good or ill—making this a genuinely special and precarious moment.

4. Scientific Wonder Grows with Knowledge, Not Despite It

Contrary to those who claim science diminishes mystery, Rees - the co-author of an upcoming book about scientific wonder - argues that "the more we understand, the more wonderful and complicated things appear." As knowledge advances, new mysteries emerge that couldn't even be conceived decades earlier.

5. Humility About Human Limitations Is Essential

Just as "a monkey can't understand quantum mechanics," there may be fundamental aspects of reality beyond human comprehension. Rees warns against immediately invoking God for unexplained phenomena, advocating instead for accepting our cognitive limits while continuing to push boundaries.

Keen On America is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit keenon.substack.com/subscribe

Alex Kurtzman