Capture of 'AI Safety' Terminology

Philosopher Dr. David Thorstad tears into one of AI safety's most influential arguments: the singularity hypothesis. We discuss why the idea of recursive self-improvement leading to superintelligence doesn't hold up under scrutiny, how these arguments have redirected hundreds of millions in funding away from proven interventions, and why people keep backpedaling to weaker versions when challenged.

David walks through the actual structure of singularity arguments, explains why similar patterns show up in other longtermist claims, and makes the case for why we should focus on concrete problems happening right now like poverty, disease, the rise of authoritarianism instead of speculative far-future scenarios.

Chapters

• (00:00) - Intro
• (02:13) - David's background
• (08:00) - (Against) The Singularity Hypothesis
• (29:46) - Beyond the The Singularity
• (39:56) - What We Should Actually Be Worried About
• (49:00) - Philanthropic Funding

• David's personal website
• Reflective Altruism, David's blog

The Singularity Hypothesis

• David's Philosophical Studies article - Against the singularity hypothesis
• Time "AI Dictionary" page - Singularity
• EA Forum blogpost - Summary: Against the singularity hypothesis
• Journal of Conciousness Studies article - The Singularity: A Philisophical Analysis
• Interim Report from the Panel Chairs: AAAI Presidential Panel on Long-Term AI Futures
• Epoch AI blogpost - Do the returns to software R&D point towards a singularity?
• Epoch AI report - Estimating Idea Production: A Methodological Survey

Funding References

• LessWrong blogpost - An Overview of the AI Safety Funding Situation
• AISafety.com funding page
• Report - Stanford AI Index 2025, Chapter 4.3
• Forbes article - AI Spending To Exceed A Quarter Trillion Next Year
• AI Panic article - The “AI Existential Risk” Industrial Complex
• GiveWell webpage - How Much Does It Cost To Save a Life?
• Wikipedia article - Purchasing power parity

Pascal's Mugging and the St. Petersburg Paradox

• Wikipedia article - St. Petersburg Paradox
• Conjecture Magazine article - Pascal’s Mugging and Bad Explanations
• neurabites explainer - Ergodicity: the Most Over-Looked Assumption
• Wikipedia article - Extraordinary claims require extraordinary evidence

The Time of Perils

• Global Priorities Institute working paper - Existential risk pessimism and the time of perils
• Ethics article - Mistakes in the Moral Mathematics of Existential Risk
• Philosophy & Public Affairs article - High Risk, Low Reward: A Challenge to the Astronomical Value of Existential Risk Mitigation
• Toby Ord Book - The Precipice
• Rethink Priorities blogpost - Charting the precipice
• AI Futures Project blogpost - AI 2027

Trump's Higher Education Threat Compact

• Wikipedia article - Compact for Academic Excellence in Higher Education
• Pen America explainer - What is Trump’s Compact for Higher Education? And More Frequently Asked Questions
• Statement by the Vanderbilt AAUP Executive Committee on the “Compact for Academic Excellence in Higher Education”
• The Vanderbilt Hustler article - BREAKING: Chancellor Daniel Diermeier fails to reject higher education compact, reaffirms Vanderbilt’s values and openness to discussion
• The Vanderbilt Hustler article - Students and faculty organize rally outside Kirkland Hall against Trump administration’s higher education compact
• Free Speech Center article - Compact for Academic Excellence

More of David's Work

• Global Priorities Institute working paper - What power-seeking theorems do not show
• Book - Essays on Longtermism

Vibe Shift

• Blood in the Machine article - GPT-5 Is a Joke. Will It Matter?
• Futurism article - Evidence Grows That GPT-5 Is a Bit of a Dud
• Gary Marcus substack - GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
• Pew Research report - How the U.S. Public and AI Experts View Artificial Intelligence
• N...