Exploring the Challenges of the Age of Ajji 55 and the Future of AI

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Loving a world you don't trust, published by Joe Carlsmith on June 18, 2024 on LessWrong.
(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)
This is the final essay in a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essays can be read fairly well on their own, but see here for a brief summary of the series as a whole. There's also a PDF of the whole series here.
Warning: spoilers for Angels in America; and moderate spoilers for Harry Potter and the Methods of Rationality.)
"I come into the presence of still water..."
~Wendell Berry
A lot of this series has been about problems with yang - that is, with the active element in the duality of activity vs. receptivity, doing vs. not-doing, controlling vs. letting go.[1] In particular, I've been interested in the ways that "deep atheism" (that is, a fundamental mistrust towards Nature, and towards bare intelligence) can propel itself towards an ever-more yang-y, controlling relationship to Otherness, and to the universe as a whole.
I've tried to point at various ways this sort of control-seeking can go wrong in the context of AGI, and to highlight a variety of less-controlling alternatives (e.g. "gentleness," "liberalism/niceness/boundaries," and "green") that I think have a role to play.[2]
This is the final essay in the series. And because I've spent so much time on potential problems with yang, and with deep atheism, I want to close with an effort to make sure I've given both of them their due, and been clear about my overall take. To this end, the first part of the essay praises certain types of yang directly, in an effort to avoid over-correction towards yin.
The second part praises something quite nearby to deep atheism that I care about a lot - something I call "humanism." And the third part tries to clarify the depth of atheism I ultimately endorse. In particular, I distinguish between trust in the Real, and various other attitudes towards it - attitudes like love, reverence, loyalty, and forgiveness. And I talk about ways these latter attitudes can still look the world's horrors in the eye.
In praise of yang
Let's start with some words in praise of yang.
In praise of black
Recall "black," from my essay on green. Black, on my construal of the colors, is the color for power, effectiveness, instrumental rationality - and hence, perhaps, the color most paradigmatically associated with yang. And insofar as I was especially interested in green qua yin, black was green's most salient antagonist.
So I want to be clear: I think black is great.[3] Or at least, some aspects of it. Not black qua ego. Not black that wants power and domination for its sake.[4] Rather: black as the color of not fucking around. Of cutting through the bullshit; rejecting what Lewis calls "soft soap"; refusing to pretend things are prettier, or easier, or more comfortable; holding fast to the core thing.
I wrote, in my essay on sincerity, about the idea of "seriousness." Black, I think, is the most paradigmatically serious color.
And it's the color of what Yudkowsky calls "the void" - that nameless, final virtue of rationality; the one that carries your movement past your map, past the performance of effort, and into contact with the true goal.[5] Yudkowsky cites Miyamoto Musashi:
The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means... If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.
Musashi (image source here)
In this sense, I think, black is the color of actually caring. That is: one becomes serious, centrally, when there are stak...