Understanding Extinction Risk through Defense Layers

From one point of view academia forms one big 'epistemic' system — a process which directs attention, generates ideas, and judges which are good. Traditional print media is another such system, and we can think of society as a whole as a huge epistemic system, made up of these and many other subsystems.

 How these systems absorb, process, combine and organise information will have a big impact on what humanity as a whole ends up doing with itself — in fact, at a broad level it basically entirely determines the direction of the future.

 With that in mind, today’s guest Owen Cotton-Barratt has founded the Research Scholars Programme (RSP) at the Future of Humanity Institute at Oxford University, which gives early-stage researchers leeway to try to understand how the world works.

Links to learn more, summary and full transcript.

Instead of you having to pay for a masters degree, the RSP pays *you* to spend significant amounts of time thinking about high-level questions, like "What is important to do?” and “How can I usefully contribute?"

Participants get to practice their research skills, while also thinking about research as a process and how research communities can function as epistemic systems that plug into the rest of society as productively as possible.

The programme attracts people with several years of experience who are looking to take their existing knowledge — whether that’s in physics, medicine, policy work, or something else — and apply it to what they determine to be the most important topics.

It also attracts people without much experience, but who have a lot of ideas. If you went directly into a PhD programme, you might have to narrow your focus quickly. But the RSP gives you time to explore the possibilities, and to figure out the answer to the question “What’s the topic that really matters, and that I’d be happy to spend several years of my life on?”

Owen thinks one of the most useful things about the two-year programme is being around other people — other RSP participants, as well as other researchers at the Future of Humanity Institute — who are trying to think seriously about where our civilisation is headed and how to have a positive impact on this trajectory.

Instead of being isolated in a PhD, you’re surrounded by folks with similar goals who can push back on your ideas and point out where you’re making mistakes. Saving years not pursuing an unproductive path could mean that you will ultimately have a much bigger impact with your career.

RSP applications are set to open in the Spring of 2021 — but Owen thinks it’s helpful for people to think about it in advance.

In today’s episode, Arden and Owen mostly talk about Owen’s own research. They cover:

• Extinction risk classification and reduction strategies
• Preventing small disasters from becoming large disasters
• How likely we are to go from being in a collapsed state to going extinct
• What most people should do if longtermism is true
• Advice for mathematically-minded people
• And much more

Chapters:
 • Rob’s intro (00:00:00)
• The interview begins (00:02:22)
• Extinction risk classification and reduction strategies (00:06:02)
• Defense layers (00:16:37)
• Preventing small disasters from becoming large disasters (00:23:31)
• Risk factors (00:38:57)
• How likely are we to go from being in a collapsed state to going extinct? (00:48:02)
• Estimating total levels of existential risk (00:54:35)
• Everyday longtermism (01:01:35)
• What should most people do if longtermism is true? (01:12:18)
• 80,000 Hours’ issue with promoting career paths (01:24:12)
• The existential risk of making a lot of really bad decisions (01:29:27)
• What should longtermists do differently today (01:39:08)
• Biggest concerns with this framework (01:51:28)
• Research careers (02:04:04)
• Being a mathematician (02:13:33)
• Advice for mathematically minded people (02:24:30)
• Rob’s outro (02:37:32)

 Producer: Keiran Harris
 Audio mastering: Ben Cordell
 Transcript: Zakee Ulhaq