Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?

By Yoshua Bengio et al.

This paper argues that building generalist AI agents poses catastrophic risks, from misuse by bad actors to a potential loss of human control. As an alternative, the authors propose “Scientist AI,” a non-agentic system designed to explain the world through theory generation and question-answering rather than acting in it. They suggest this path could accelerate scientific progress, including in AI safety, while avoiding the dangers of agency-driven AI.

Source:

https://arxiv.org/pdf/2502.15657

A podcast by BlueDot Impact.

Learn more on the AI Safety Fundamentals website.