In this book, Daniel Ellsberg chronicles his experiences as a nuclear war planner during the Cold War, revealing shocking details about the delegation of authority to initiate nuclear attacks, secret plans for general nuclear war, and the precarious nature of nuclear strategy. The book is divided into two parts: the first part is a memoir/confession of Ellsberg's intellectual and moral evolution, while the second part adopts a more academic perspective, analyzing the principles of 'just war' policy and the consequences of nuclear exchanges. Ellsberg argues that little has changed in nuclear policy since the Eisenhower and Kennedy years and offers feasible steps to dismantle the 'doomsday machine' and avoid nuclear catastrophe.
In this book, Toby Ord argues that humanity is in a uniquely dangerous period, which he terms 'the Precipice,' beginning with the first atomic bomb test in 1945. Ord examines various existential risks, including natural and anthropogenic threats, and estimates that there is a one in six chance of humanity suffering an existential catastrophe within the next 100 years. He advocates for a major reorientation in how we see the world and our role in it, emphasizing the need for collective action to minimize these risks and ensure a safe future for humanity. The book integrates insights from multiple disciplines, including physics, biology, earth science, computer science, history, anthropology, statistics, international relations, and moral philosophy[1][3][5].
Preventing the apocalypse may sound like an idiosyncratic activity, and it sometimes is justified on exotic grounds, such as the potential for humanity to become a galaxy-spanning civilisation.
But the policy of US government agencies is already to spend up to $4 million to save the life of a citizen, making the death of all Americans a $1,300,000,000,000,000 disaster.
According to Carl Shulman, research associate at Oxford University’s Future of Humanity Institute, that means you don’t need any fancy philosophical arguments about the value or size of the future to justify working to reduce existential risk — it passes a mundane cost-benefit analysis whether or not you place any value on the long-term future.
Rebroadcast: this episode was originally released in October 2021.
Links to learn more, summary, and full transcript.
The key reason to make it a top priority is factual, not philosophical. That is, the risk of a disaster that kills billions of people alive today is alarmingly high, and it can be reduced at a reasonable cost. A back-of-the-envelope version of the argument runs:
- The US government is willing to pay up to $4 million (depending on the agency) to save the life of an American.
- So saving all US citizens at any given point in time would be worth $1,300 trillion.
- If you believe that the risk of human extinction over the next century is something like one in six (as Toby Ord suggests is a reasonable figure in his book The Precipice), then it would be worth the US government spending up to $2.2 trillion to reduce that risk by just 1%, in terms of American lives saved alone.
- Carl thinks it would cost a lot less than that to achieve a 1% risk reduction if the money were spent intelligently. So it easily passes a government cost-benefit test, with a very big benefit-to-cost ratio — likely over 1000:1 today.
This argument helped NASA get funding to scan the sky for any asteroids that might be on a collision course with Earth, and it was directly promoted by famous economists like Richard Posner, Larry Summers, and Cass Sunstein.
If the case is clear enough, why hasn’t it already motivated a lot more spending or regulations to limit existential risks — enough to drive down what any additional efforts would achieve?
Carl thinks that one key barrier is that infrequent disasters are rarely politically salient. Research indicates that extra money is spent on flood defences in the years immediately following a massive flood — but as memories fade, that spending quickly dries up. Of course the annual probability of a disaster was the same the whole time; all that changed is what voters had on their minds.
Carl suspects another reason is that it’s difficult for the average voter to estimate and understand how large these respective risks are, and what responses would be appropriate rather than self-serving. If the public doesn’t know what good performance looks like, politicians can’t be given incentives to do the right thing.
It’s reasonable to assume that if we found out a giant asteroid were going to crash into the Earth one year from now, most of our resources would be quickly diverted into figuring out how to avert catastrophe.
But even in the case of COVID-19, an event that massively disrupted the lives of everyone on Earth, we’ve still seen a substantial lack of investment in vaccine manufacturing capacity and other ways of controlling the spread of the virus, relative to what economists recommended.
Carl expects that all the reasons we didn’t adequately prepare for or respond to COVID-19 — with excess mortality over 15 million and costs well over $10 trillion — bite even harder when it comes to threats we’ve never faced before, such as engineered pandemics, risks from advanced artificial intelligence, and so on.
Today’s episode is in part our way of trying to improve this situation. In today’s wide-ranging conversation, Carl and Rob also cover:
- A few reasons Carl isn’t excited by ‘strong longtermism’
- How x-risk reduction compares to GiveWell recommendations
- Solutions for asteroids, comets, supervolcanoes, nuclear war, pandemics, and climate change
- The history of bioweapons
- Whether gain-of-function research is justifiable
- Successes and failures around COVID-19
- The history of existential risk
- And much more
Producer: Keiran Harris
Audio mastering: Ben Cordell
Transcriptions: Katy Moore
80,000 Hours Podcast 
