Paperclipper Origins and Two Kinds

There has been a lot of talk about "p(doom)"over the last few years. This has always rubbed me the wrong waybecause "p(doom)" didn't feel like it mapped to any specific belief in my head.In private conversations I'd sometimes give my p(doom) as 12%, with the caveatthat "doom" seemed nebulous and conflated between several different concepts.At some point it was decideda p(doom) over 10% makes you a "doomer" because it means what actions you should take with respect toAI are overdetermined. I did not and do not feel that is true. But any time Ifelt prompted to explain my position I'd find I could explain a little bit ofthis or that, but not really convey the whole thing. As it turns out doom hasa lot of parts, and every part is entangled with every other part so no matterwhich part you explain you always feel like you're leaving the crucial parts out. Doom ismore like an onion than asingle event, a distribution over AI outcomes people frequentlyrespond to with the force of the fear of death. Some of these outcomes are lessthan death and some [...]

---

Outline:

(03:46) 1. Existential Ennui

(06:40) 2. Not Getting Immortalist Luxury Gay Space Communism

(13:55) 3. Human Stock Expended As Cannon Fodder Faster Than Replacement

(19:37) 4. Wiped Out By AI Successor Species

(27:57) 5. The Paperclipper

(42:56) Would AI Successors Be Conscious Beings?

(44:58) Would AI Successors Care About Each Other?

(49:51) Would AI Successors Want To Have Fun?

(51:11) VNM Utility And Human Values

(55:57) Would AI successors get bored?

(01:00:16) Would AI Successors Avoid Wireheading?

(01:06:07) Would AI Successors Do Continual Active Learning?

(01:06:35) Would AI Successors Have The Subjective Experience of Will?

(01:12:00) Multiply

(01:15:07) 6. Recipes For Ruin

(01:18:02) Radiological and Nuclear

(01:19:19) Cybersecurity

(01:23:00) Biotech and Nanotech

(01:26:35) 7. Large-Finite Damnation

---

First published:
November 17th, 2025

Source:
https://www.lesswrong.com/posts/apHWSGDiydv3ivmg6/varieties-of-doom

---



Narrated by TYPE III AUDIO.

---