80,000 Hours Podcast AGI disagreements and misconceptions: Rob, Luisa, & past guests hash it out
03:12:24
The Future of Humanity
- Humanity will either go extinct, become superhuman, or things stay the same.
- The least likely scenario is that everything stays more or less the same.
Overrated AGI Worries
- Ajeya Cotra believes that AI will likely understand human values well.
- She's more concerned about AI deceptiveness due to strong human psychology models.
AI Takeover with Messy Goals
- Ajeya Cotra thinks AI might not need simple utility functions for takeover.
- Messy psychologies and inconsistent goals, like humans, could be enough.
Get the Snipd Podcast app to discover more snips from this episode
Get the app 1 chevron_right 2 chevron_right 3 chevron_right 4 chevron_right 5 chevron_right 6 chevron_right 7 chevron_right 8 chevron_right 9 chevron_right 10 chevron_right 11 chevron_right 12 chevron_right 13 chevron_right 14 chevron_right 15 chevron_right 16 chevron_right 17 chevron_right
Intro
00:00 • 4min
Evolving Insights on AI
03:51 • 16min
Navigating the Future of Agentic AI
19:44 • 19min
Navigating AI Risks and Humanity's Future
39:05 • 13min
Navigating the Landscape of AI Risks and Misconceptions
52:22 • 3min
Navigating the Risks of AI Development: Responsible Scaling Policies
55:51 • 6min
Exploring Misconceptions in Machine Learning and AI Development
01:01:35 • 6min
Shifting Perspectives on AGI Capabilities and Development
01:07:23 • 2min
Navigating AI's Evolution and Alignment
01:09:18 • 9min
Challenging the End of History Fallacy: Historical Patterns of Change
01:18:35 • 2min
AI's Economic Impact: Optimism vs. Skepticism
01:20:28 • 6min
AI's Economic Impact and Societal Choices
01:26:26 • 25min
The Dynamics of Misinformation and Belief
01:51:41 • 17min
Navigating the AGI Paradox
02:09:09 • 6min
Consciousness, Computation, and Artificial Sentience
02:15:15 • 26min
AI's Dual Role in Animal Welfare
02:40:58 • 9min
Navigating AGI Development and Implications
02:49:36 • 22min
1984
None
George Orwell
Published in 1949, '1984' is a cautionary tale by George Orwell that explores the dangers of totalitarianism. The novel is set in a dystopian future where the world is divided into three super-states, with the protagonist Winston Smith living in Oceania, ruled by the mysterious and omnipotent leader Big Brother. Winston works at the Ministry of Truth, where he rewrites historical records to conform to the Party's ever-changing narrative. He begins an illicit love affair with Julia and starts to rebel against the Party, but they are eventually caught and subjected to brutal torture and indoctrination. The novel highlights themes of government surveillance, manipulation of language and history, and the suppression of individual freedom and independent thought.
Why the West Rules - For Now
Ian Morris
In this book, Ian Morris explores 50,000 years of human history to understand why the West has come to dominate the world. He argues that it is not differences in race, culture, or individual achievements that explain Western dominance, but rather the effects of geography on the everyday efforts of ordinary people. Morris uses a four-factor analytical tool—energy capture, urbanization, information technology, and war-making capacity—to measure social development in both the East and West. He concludes that geography explains regional differences and predicts significant changes in global power dynamics in the coming centuries due to ongoing interactions between geography and human ingenuity.
What We Owe the Future
William MacAskill
In this book, William MacAskill advocates for longtermism, the idea that positively influencing the long-term future is a key moral priority. He argues that future people count, there could be many of them, and we can make their lives better. MacAskill discusses various threats to humanity, including climate change, AI misalignment, and pandemics, and proposes strategies to ensure civilization's survival and improve its trajectory. The book explores moral and philosophical issues surrounding longtermism, including the risks of human extinction, civilizational collapse, and technological stagnation, while offering a measured optimism about the future's potential for human flourishing[1][5][4].
Will LLMs soon be made into autonomous agents? Will they lead to job losses? Is AI misinformation overblown? Will it prove easy or hard to create AGI? And how likely is it that it will feel like something to be a superhuman AGI?
With AGI back in the headlines, we bring you 15 opinionated highlights from the show addressing those and other questions, intermixed with opinions from hosts Luisa Rodriguez and Rob Wiblin recorded back in 2023.
Check out the full transcript on the 80,000 Hours website.
You can decide whether the views we expressed (and those from guests) then have held up these last two busy years. You’ll hear:
- Ajeya Cotra on overrated AGI worries
- Holden Karnofsky on the dangers of aligned AI, why unaligned AI might not kill us, and the power that comes from just making models bigger
- Ian Morris on why the future must be radically different from the present
- Nick Joseph on whether his companies internal safety policies are enough
- Richard Ngo on what everyone gets wrong about how ML models work
- Tom Davidson on why he believes crazy-sounding explosive growth stories… and Michael Webb on why he doesn’t
- Carl Shulman on why you’ll prefer robot nannies over human ones
- Zvi Mowshowitz on why he’s against working at AI companies except in some safety roles
- Hugo Mercier on why even superhuman AGI won’t be that persuasive
- Rob Long on the case for and against digital sentience
- Anil Seth on why he thinks consciousness is probably biological
- Lewis Bollard on whether AI advances will help or hurt nonhuman animals
- Rohin Shah on whether humanity’s work ends at the point it creates AGI
And of course, Rob and Luisa also regularly chime in on what they agree and disagree with.
Chapters:
- Cold open (00:00:00)
- Rob's intro (00:00:58)
- Rob & Luisa: Bowerbirds compiling the AI story (00:03:28)
- Ajeya Cotra on the misalignment stories she doesn’t buy (00:09:16)
- Rob & Luisa: Agentic AI and designing machine people (00:24:06)
- Holden Karnofsky on the dangers of even aligned AI, and how we probably won’t all die from misaligned AI (00:39:20)
- Ian Morris on why we won’t end up living like The Jetsons (00:47:03)
- Rob & Luisa: It’s not hard for nonexperts to understand we’re playing with fire here (00:52:21)
- Nick Joseph on whether AI companies’ internal safety policies will be enough (00:55:43)
- Richard Ngo on the most important misconception in how ML models work (01:03:10)
- Rob & Luisa: Issues Rob is less worried about now (01:07:22)
- Tom Davidson on why he buys the explosive economic growth story, despite it sounding totally crazy (01:14:08)
- Michael Webb on why he’s sceptical about explosive economic growth (01:20:50)
- Carl Shulman on why people will prefer robot nannies over humans (01:28:25)
- Rob & Luisa: Should we expect AI-related job loss? (01:36:19)
- Zvi Mowshowitz on why he thinks it’s a bad idea to work on improving capabilities at cutting-edge AI companies (01:40:06)
- Holden Karnofsky on the power that comes from just making models bigger (01:45:21)
- Rob & Luisa: Are risks of AI-related misinformation overblown? (01:49:49)
- Hugo Mercier on how AI won’t cause misinformation pandemonium (01:58:29)
- Rob & Luisa: How hard will it actually be to create intelligence? (02:09:08)
- Robert Long on whether digital sentience is possible (02:15:09)
- Anil Seth on why he believes in the biological basis of consciousness (02:27:21)
- Lewis Bollard on whether AI will be good or bad for animal welfare (02:40:52)
- Rob & Luisa: The most interesting new argument Rob’s heard this year (02:50:37)
- Rohin Shah on whether AGI will be the last thing humanity ever does (02:57:35)
- Rob's outro (03:11:02)
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Transcriptions and additional content editing: Katy Moore