Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Third-party testing as a key ingredient of AI policy, published by Zac Hatfield-Dodds on March 25, 2024 on The AI Alignment Forum.
(nb: this post is written for anyone interested, not specifically aimed at this forum)
We believe that the AI sector needs effective third-party testing for frontier AI systems. Developing a testing regime and associated policy interventions based on the insights of industry, government, and academia is the best way to avoid societal harm - whether deliberate or accidental - from AI systems.
Our deployment of large-scale, generative AI systems like Claude has shown us that work is needed to set up the policy environment to respond to the capabilities of today's most powerful AI models, as well as those likely to be built in the future. In this post, we discuss what third-party testing looks like, why it's needed, and describe some of the research we've done to arrive at this policy position.
We also discuss how ideas around testing relate to other topics on AI policy, such as openly accessible models and issues of regulatory capture.
Policy overview
Today's frontier AI systems demand a third-party oversight and testing regime to validate their safety. In particular, we need this oversight for understanding and analyzing model behavior relating to issues like election integrity, harmful discrimination, and the potential for national security misuse.
We also expect more powerful systems in the future will demand deeper oversight - as discussed in our 'Core views on AI safety' post, we think there's a chance that today's approaches to AI development could yield systems of immense capability, and we expect that increasingly powerful systems will need more expansive testing procedures.
A robust, third-party testing regime seems like a good way to complement sector-specific regulation as well as develop the muscle for policy approaches that are more general as well.
Developing a third-party testing regime for the AI systems of today seems to give us one of the best tools to manage the challenges of AI today, while also providing infrastructure we can use for the systems of the future.
We expect that ultimately some form of third-party testing will be a legal requirement for widely deploying AI models, but designing this regime and figuring out exactly what standards AI systems should be assessed against is something we'll need to iterate on in the coming years - it's not obvious what would be appropriate or effective today, and the way to learn that is to prototype such a regime and generate evidence about it.
An effective third-party testing regime will:
Give people and institutions more trust in AI systems
Be precisely scoped, such that passing its tests is not so great a burden that small companies are disadvantaged by them
Be applied only to a narrow set of the most computationally-intensive, large-scale systems; if implemented correctly, the vast majority of AI systems would not be within the scope of such a testing regime
Provide a means for countries and groups of countries to coordinate with one another via developing shared standards and experimenting with Mutual Recognition agreements
Such a regime will have the following key ingredients [1]:
Effective and broadly-trusted tests for measuring the behavior and potential misuses of a given AI system
Trusted and legitimate third-parties who can administer these tests and audit company testing procedures
Why we need an effective testing regime
This regime is necessary because frontier AI systems - specifically, large-scale generative models that consume substantial computational resources - don't neatly fit into the use-case and sector-specific frameworks of today.
These systems are designed to be 'everything machines' - Gemini, ChatGPT, and Claude can all be adapted to a vast number of do...
The Nonlinear Library