2min chapter

#112 – Carl Shulman on the common-sense case for existential risk work and its practical implications

80,000 Hours Podcast

notes

CHAPTER

The History of Bio-Protection Research

The soviet bya weapons programme was a vast majority of all the people who have ever worked on bi weapons. They had as many as, my estimates vary, but as many as 50 thousand people working in that programme. The thing that scares me the most about that history is the rate of accidental escapes and infections from that sort of work.

00:00

Transcript

Episode notes

Preventing the apocalypse may sound like an idiosyncratic activity, and it sometimes is justified on exotic grounds, such as the potential for humanity to become a galaxy-spanning civilisation.

But the policy of US government agencies is already to spend up to $4 million to save the life of a citizen, making the death of all Americans a $1,300,000,000,000,000 disaster.

According to Carl Shulman, research associate at Oxford University's Future of Humanity Institute, that means you don’t need any fancy philosophical arguments about the value or size of the future to justify working to reduce existential risk — it passes a mundane cost-benefit analysis whether or not you place any value on the long-term future.

Links to learn more, summary and full transcript.

The key reason to make it a top priority is factual, not philosophical. That is, the risk of a disaster that kills billions of people alive today is alarmingly high, and it can be reduced at a reasonable cost. A back-of-the-envelope version of the argument runs:
• The US government is willing to pay up to $4 million (depending on the agency) to save the life of an American.
• So saving all US citizens at any given point in time would be worth $1,300 trillion.
• If you believe that the risk of human extinction over the next century is something like one in six (as Toby Ord suggests is a reasonable figure in his book The Precipice), then it would be worth the US government spending up to $2.2 trillion to reduce that risk by just 1%, in terms of American lives saved alone.
• Carl thinks it would cost a lot less than that to achieve a 1% risk reduction if the money were spent intelligently. So it easily passes a government cost-benefit test, with a very big benefit-to-cost ratio — likely over 1000:1 today.

This argument helped NASA get funding to scan the sky for any asteroids that might be on a collision course with Earth, and it was directly promoted by famous economists like Richard Posner, Larry Summers, and Cass Sunstein.

If the case is clear enough, why hasn't it already motivated a lot more spending or regulations to limit existential risks — enough to drive down what any additional efforts would achieve?

Carl thinks that one key barrier is that infrequent disasters are rarely politically salient. Research indicates that extra money is spent on flood defences in the years immediately following a massive flood — but as memories fade, that spending quickly dries up. Of course the annual probability of a disaster was the same the whole time; all that changed is what voters had on their minds.

Carl expects that all the reasons we didn’t adequately prepare for or respond to COVID-19 — with excess mortality over 15 million and costs well over $10 trillion — bite even harder when it comes to threats we've never faced before, such as engineered pandemics, risks from advanced artificial intelligence, and so on.

Today’s episode is in part our way of trying to improve this situation. In today’s wide-ranging conversation, Carl and Rob also cover:
• A few reasons Carl isn't excited by 'strong longtermism'
• How x-risk reduction compares to GiveWell recommendations
• Solutions for asteroids, comets, supervolcanoes, nuclear war, pandemics, and climate change
• The history of bioweapons
• Whether gain-of-function research is justifiable
• Successes and failures around COVID-19
• The history of existential risk
• And much more

Chapters: