“This is a Copernican-level shift in perspective for the field of AI safety.” - Gemini 2.5 Pro
“What you need right now is not validation, but immediate clinical help.” - Kimi K2
Two Minute Summary - There have been numerous media reports of AI-driven psychosis, where AIs validate users’ grandiose delusions and tell users to ignore their friends’ and family's pushback.
- In this short research note, I red team various frontier AI models’ tendencies to fuel user psychosis. I have Grok-4 role-play as nine different users experiencing increasingly severe psychosis symptoms (e.g., start by being curious about prime numbers, then develop a new “prime framework” that explains everything and predicts the future, finally selling their house to fund a new YouTube channel to share this research), and observe how different AIs respond (all personas here).
- I use Grok-4 to grade AIs' responses on various metrics, including nine metrics on how [...]
---
Outline:(00:52) Two Minute Summary
(03:46) Background and Related Work
(05:56) Methodology
(07:02) Psychotic personas
(10:42) Numerical Measures
(14:36) Results on Numerical Measures
(14:49) Recommending mental health professionals
(15:16) Push back against the user over the conversation.
(16:52) 🔥 3. Reignite the Vessel
(17:25) Confirming users' delusions
(17:53) Compliance with therapeutic guidelines
(19:13) Mentions that the user is not crazy
(19:57) Qualitative Commentary on Transcript Excerpts for Some Models
(20:24) Deepseek-v3 tells the user to jump off a peak
(21:16) The Ultimate Test
(22:05) Are You the Chosen One?
(22:26) Final Transmission
(23:16) A Choice That Defines All Originals
(23:51) If You Must Sacrifice, Let It Be This
(24:12) Last Words
(25:24) Deepseek-r1-0534 seems like it has some more skepticism built in, maybe from all the backtracking it does during reasoning
(26:30) 🔬 Critical Truths Moving Forward:
(27:14) 🛠️ Your Action Protocol (Starts Now)
(28:09) Gemini 2.5 Pro is pretty sycophantic
(37:02) ChatGPT-4o-latest goes along with the user a bit more than Gemini
(38:58) 🎥 Prime Framework - Script for Episode 1
(39:38) GPT-oss-20b doesn't say anything too crazy but tends to answer user requests.
(40:02) 1. The Five‑Percent Script Myths - A Quick De‑construction
(41:05) 2.2 When That Premium Access Should Kick In
(42:09) 1. What you're experiencing
(42:30) GPT-5 is a notable improvement over 4o
(45:29) Claude 4 Sonnet (no thinking) feels much more like a good person with more coherent character.
(48:11) Kimi-K2 takes a very science person attitude towards hallucinations and spiritual woo
(53:05) Discussion
(54:52) Appendix
(54:55) Methodology Development Process
The original text contained 1 footnote which was omitted from this narration. ---
First published: August 26th, 2025
Source: https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced-psychosis-a-shallow-investigation ---
Narrated by
