"Thoughts on the AI Safety Summit company policy requests and responses" by So8res
Nov 3, 2023
auto_awesome
Amazon, Anthropic, DeepMind, Inflection, Meta, Microsoft, and OpenAI outline their AI Safety Policies. The UK government's requests are analyzed, with missing priorities and organizations that excel identified. Topics discussed include preventing model misuse, responsible capability scaling, addressing emerging risks in AGI development, and ranking AI safety policies of various companies. The importance of monitoring risks and evaluating proposals for monitoring risks and benefits is also explored.
Governments should prioritize steps like independent risk assessments and computing thresholds to address existential risks from AI.
When assessing AI safety policies, it is crucial to consider the track record and overall behavior of companies.
Deep dives
Evaluating AI Safety Categories
The podcast discusses the nine areas outlined by the UK government when requesting AI safety policies from seven companies. The host provides insights on each category, highlighting the importance of responsible capability scaling, model evaluations, model reporting, security controls, reporting structure for vulnerabilities, identifiers of AI-generated content, research on risks, preventing model misuse, and data input controls.
Higher Priorities for Governments
The host suggests that while asking companies to create AI safety policies is useful, there are other steps governments should prioritize to address existential risks from AI. These include setting compute and capabilities thresholds, centralizing and monitoring chips, halting improved chip development, establishing independent risk assessments, and planning for scenarios when emerging risks cannot be effectively addressed.
Assessment of AI Safety Policies
The host provides an assessment of the AI safety policies submitted by individual companies. Anthropic and OpenAI receive the highest rankings, while DeepMind's policy is considered worse. The host discusses the policies in terms of responsible scaling, alignment research, security controls, and safe deployment, emphasizing the need to consider the track record and overall behavior of companies.
Over the next two days, the UK government is hosting an AI Safety Summit focused on “the safe and responsible development of frontier AI”. They requested that seven companies (Amazon, Anthropic, DeepMind, Inflection, Meta, Microsoft, and OpenAI) “outline their AI Safety Policies across nine areas of AI Safety”.
Below, I’ll give my thoughts on the nine areas the UK government described; I’ll note key priorities that I don’t think are addressed by company-side policy at all; and I’ll say a few words (with input from Matthew Gray, whose discussions here I’ve found valuable) about the individual companies’ AI Safety Policies.[1]
My overall take on the UK government’s asks is: most of these are fine asks; some things are glaringly missing, like independent risk assessments.
My overall take on the labs’ policies is: none are close to adequate, but some are importantly better than others, and most of the organizations are doing better than sheer denial of the primary risks.