David Duvenaud is a professor of Computer Science at the University of Toronto, co-director of the Schwartz Reisman Institute for Technology and Society, former Alignment Evals Team Lead at Anthropic, an award-winning machine learning researcher, and a close collaborator of Dr. Geoffrey Hinton. He recently co-authored Gradual Disempowerment.
We dive into David’s impressive career, his high P(Doom), his recent tenure at Anthropic, his views on gradual disempowerment, and the critical need for improved governance and coordination on a global scale.
00:00 Introducing David
03:03 Joining Anthropic and AI Safety Concerns
35:58 David’s Background and Early Influences
45:11 AI Safety and Alignment Challenges
54:08 What’s Your P(Doom)™
01:06:44 Balancing Productivity and Family Life
01:10:26 The Hamming Question: Are You Working on the Most Important Problem?
01:16:28 The PauseAI Movement
01:20:28 Public Discourse on AI Doom
01:24:49 Courageous Voices in AI Safety
01:43:54 Coordination and Government Role in AI
01:47:41 Cowardice in AI Leadership
02:00:05 Economic and Existential Doom
02:06:12 Liron’s Post-Show
Show Notes
David’s Twitter — https://x.com/DavidDuvenaud
Schwartz Reisman Institute for Technology and Society — https://srinstitute.utoronto.ca/
Jürgen Schmidhuber’s Home Page — https://people.idsia.ch/~juergen/
Ryan Greenblatt's LessWrong comment about a future scenario where there's a one-time renegotiation of power and heat from superintelligent AI projects causes the oceans to boil: https://www.lesswrong.com/posts/pZhEQieM9otKXhxmd/gradual-disempowerment-systemic-existential-risks-from?commentId=T7KZGGqq2Z4gXZsty
Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligence
PauseAI, the volunteer organization I’m part of: https://pauseai.info
Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit
lironshapira.substack.com
