Today I’m sharing my interview on Robert Wright’s Nonzero Podcast from last May. Rob is an especially sharp interviewer who doesn't just nod along, he had great probing questions for me.
This interview happened right after Ilya Sutskever and Jan Leike resigned from OpenAI in May 2024, continuing a pattern that goes back to Dario Amodei leaving to start Anthropic. These aren't fringe doomers; these are the people hired specifically to solve the safety problem, and they keep concluding it's not solvable at the current pace.
00:00:00 - Liron’s preface
00:02:10 - Robert Wright introduces Liron
00:04:02 - PauseAI protests at OpenAI headquarters
00:05:15 - OpenAI resignations (Ilya Sutskever, Jan Leike, Dario Amodei, Paul Christiano, Daniel Kokotajlo)
00:15:30 - P vs NP problem as analogy for AI alignment difficulty
00:22:31 - AI pause movement and protest turnout
00:29:02 - Defining AI doom and sci-fi scenarios
00:32:05 - What’s My P(Doom)™
00:35:18 - Fast vs slow AI takeoff and Sam Altman's position
00:42:33 - Paperclip thought experiment and instrumental convergence explanation
00:54:40 - Concrete examples of AI power-seeking behavior (business assistant scenario)
01:00:58 - GPT-4 TaskRabbit deception example and AI reasoning capabilities
01:09:00 - AI alignment challenges and human values discussion
01:17:33 - Wrap-up and transition to premium subscriber content
Show Notes
This episode on Rob’s Nonzero Newsletter. You can subscribe for premium access to the last 1 hour of our discussion! — https://www.nonzero.org/p/in-defense-of-ai-doomerism-robert
This episode on Rob’s YouTube — https://www.youtube.com/watch?v=VihA_-8kBNg
PauseAI — https://pauseai.info
PauseAI US — http://pauseai-us.org
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit
lironshapira.substack.com
