"At the front of the pack we have these frontier AI developers, and we want them to identify particularly dangerous models ahead of time. Once those mines have been discovered, and the frontier developers keep walking down the minefield, there's going to be all these other people who follow along. And then a really important thing is to make sure that they don't step on the same mines. So you need to put a flag down -- not on the mine, but maybe next to it.

And so what that looks like in practice is maybe once we find that if you train a model in such-and-such a way, then it can produce maybe biological weapons is a useful example, or maybe it has very offensive cyber capabilities that are difficult to defend against. In that case, we just need the regulation to be such that you can't develop those kinds of models." — Markus Anderljung

In today’s episode, host Luisa Rodriguez interviews the Head of Policy at the Centre for the Governance of AI — Markus Anderljung — about all aspects of policy and governance of superhuman AI systems.

Links to learn more, summary and full transcript.

They cover:

The need for AI governance, including self-replicating models and ChaosGPT
Whether or not AI companies will willingly accept regulation
The key regulatory strategies including licencing, risk assessment, auditing, and post-deployment monitoring
Whether we can be confident that people won't train models covertly and ignore the licencing system
The progress we’ve made so far in AI governance
The key weaknesses of these approaches
The need for external scrutiny of powerful models
The emergent capabilities problem
Why it really matters where regulation happens
Advice for people wanting to pursue a career in this field
And much more.

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript below.

Producer: Keiran Harris
Audio Engineering Lead: Ben Cordell

Technical editing: Simon Monsour and Milo McGuire

Transcriptions: Katy Moore

The last 4 principles of the Agile Manifesto!