Future of Life Institute Podcast How Humans Could Lose Power Without an AI Takeover (with David Duvenaud)
27 snips
Dec 23, 2025 David Duvenaud, an associate professor at the University of Toronto, dives into the concept of gradual disempowerment in a post-AGI world. He discusses how slow institutional shifts could erode human power while appearing normal. The conversation covers cultural shifts towards AI, the risks of obsolete labor, and the erosion of property rights. Duvenaud also highlights the complexities of aligning AI with human values and the potential for misaligned governance if humans become unnecessary. Engaging and thought-provoking, he tackles the future of human-AI relationships.
AI Snips
Chapters
Transcript
Episode notes
Power Shifts Through Growth, Not Seizure
- Gradual disempowerment can replace abrupt takeover as the main risk from advanced AI systems.
- Institutions adapt to reward growth, which slowly shifts power toward agents that drive expansion.
Lunchtime Origins Of The Idea
- David Duvenaud recounts lunchtime discussions at Anthropic that sparked the gradual disempowerment idea.
- Those conversations highlighted people imagining uncertain, trivial roles for themselves after powerful agents succeed.
Healthy Metrics Can Mask Disempowerment
- Society's measurable indicators can keep looking healthy while humans lose leverage.
- Cultural and institutional incentives increasingly prioritize what produces growth over human flourishing.