Meta-strategic Risks and Governance Challenges

Looking back, it appears that much of my intellectual output could be described as legibilizing work, or trying to make certain problems in AI risk more legible to myself and others. I've organized the relevant posts and comments into the following list, which can also serve as a partial guide to problems that may need to be further legibilized, especially beyond LW/rationalists, to AI researchers, funders, company leaders, government policymakers, their advisors (including future AI advisors), and the general public.

1. Philosophical problems
   1. Probability theory
   2. Decision theory
   3. Beyond astronomical waste (possibility of influencing vastly larger universes beyond our own)
   4. Interaction between bargaining and logical uncertainty
   5. Metaethics
   6. Metaphilosophy: 1, 2
2. Problems with specific philosophical and alignment ideas
   1. Utilitarianism: 1, 2
   2. Solomonoff induction
   3. "Provable" safety
   4. CEV
   5. Corrigibility
   6. IDA (and many scattered comments)
   7. UDASSA
   8. UDT
3. Human-AI safety (x- and s-risks arising from the interaction between human nature and AI design)
   1. Value differences/conflicts between humans
   2. “Morality is scary” (human morality is often the result of status games amplifying random aspects of human value, with frightening results)
   3. [...]

---

First published:
November 9th, 2025

Source:
https://www.lesswrong.com/posts/7XGdkATAvCTvn4FGu/problems-i-ve-tried-to-legibilize

---

Narrated by TYPE III AUDIO.