How can a binding international treaty be agreed and put into practice, when many parties are strongly tempted to break the rules of the agreement, for commercial or military advantage, and when cheating may be hard to detect? That’s the dilemma we’ll examine in this episode, concerning possible treaties to govern the development and deployment of advanced AI.

Our guest is Otto Barten, Director of the Existential Risk Observatory, which is based in the Netherlands but operates internationally. In November last year, Time magazine published an article by Otto, advocating what his organisation calls a Conditional AI Safety Treaty. In March this year, these ideas were expanded into a 34-page preprint which we’ll be discussing today, “International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty”.

Before co-founding the Existential Risk Observatory in 2021, Otto had roles as a sustainable energy engineer, data scientist, and entrepreneur. He has a BSc in Theoretical Physics from the University of Groningen and an MSc in Sustainable Energy Technology from Delft University of Technology.

Selected follow-ups:

Existential Risk Observatory
There Is a Solution to AI’s Existential Risk Problem - Time
International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty - Otto Barten and colleagues
The Precipice: Existential Risk and the Future of Humanity - book by Toby Ord
Grand futures and existential risk - Lecture by Anders Sandberg in London attended by Otto
PauseAI
StopAI
Responsible Scaling Policies - METR
Meta warns of 'worse' experience for European users - BBC News
Accidental Nuclear War: a Timeline of Close Calls - FLI
The Vulnerable World Hypothesis - Nick Bostrom
Semiconductor Manufacturing Optics - Zeiss
California Institute for Machine Consciousness
Tipping point for large-scale social change? Just 25 percent - Penn Today

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Slave Songs of the United States

Mentioned by

Mentioned in 1 episodes

The AI-powered Podcast Player