The Nonlinear Library EA - A Case for Superhuman Governance, using AI by Ozzie Gooen
Jun 7, 2024
17:51
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Case for Superhuman Governance, using AI, published by Ozzie Gooen on June 7, 2024 on The Effective Altruism Forum.
I believe that:
1. AI-enhanced organization governance could be a potentially huge win in the next few decades.
2. AI-enhanced governance could allow organizations to reach superhuman standards, like having an expected "99.99" reliability rate of not being corrupt or not telling lies.
3. While there are clear risks to AI-enhancement at the top levels of organizations, it's likely that most of these can be managed, assuming that the implementers are reasonable.
4. AI-enhanced governance could synchronize well with AI company regulation. These companies would be well-placed to develop innovations and could hypothetically be incentivized to do much of the work. AI-enhanced governance might be necessary to ensure that these organizations are aligned with public interests.
5. More thorough investigation here could be promising for the effective altruism community.
Within effective altruism now, there's a lot of work on governance and AI, but not much on using AI for governance. AI Governance typically focuses on using conventional strategies to oversee AI organizations, while AI Alignment research focuses on aligning AI systems. However, leveraging AI to improve human governance is an underexplored area that could complement these cause areas. You can think of it as "Organizational Alignment", as a counterpoint to "AI Alignment."
This article was written after some rough ideation I've done about this area. This isn't at all a literature review or a research agenda. That said, for those interested in this topic, here are a few posts you might find interesting.
Project ideas: Governance during explosive technological growth
The Project AI Series, by OpenMined
Safety Cases: How to Justify the Safety of Advanced AI Systems
Affirmative Safety: An Approach to Risk Management for Advanced AI
What is "AI-Assisted" Governance?
AI-Assisted Governance refers to improvements in governance that leverage artificial intelligence (AI), particularly focusing on rapidly advancing areas like Large Language Models (LLMs).
Examples methods include:
1. Monitoring politicians and executives to identify and flag misaligned or malevolent behavior, ensuring accountability and integrity.
2. Enhancing epistemics and decision-making processes at the top levels of organizations, leading to more informed and rational strategies.
3. Facilitating more effective negotiations and trades between organizations, fostering better cooperation and coordination.
4. Assisting in writing and overseeing highly secure systems, such as implementing differential privacy and formally verified, bug-free decision-automation software, for use at managerial levels.
Arguments for Governance Improvements, Generally
There's already a lot of consensus in the rationalist and effective altruist communities about the importance for governance. See the topics on
Global Governance,
AI Governance, and
Nonprofit Governance for more information.
Here are some main reasons why focusing on improving governance seems particularly promising:
Concentrated Leverage
Real-world influence is disproportionately concentrated in the hands of a relatively small number of leaders in government, business, and other pivotal institutions. This is especially true in the case of rapid AI progress. Improving the reasoning and actions of this select group is therefore perhaps the most targeted, tractable, and neglected way to shape humanity's long-term future. AI tools could offer uniquely potent levers to do so.
A lot of epistemic-enhancing work focuses on helping large populations. But some people will matter many times as much as others, and these people are often in key management positions.
Dramatic Room for Improvement
It's hard to lo...
I believe that:
1. AI-enhanced organization governance could be a potentially huge win in the next few decades.
2. AI-enhanced governance could allow organizations to reach superhuman standards, like having an expected "99.99" reliability rate of not being corrupt or not telling lies.
3. While there are clear risks to AI-enhancement at the top levels of organizations, it's likely that most of these can be managed, assuming that the implementers are reasonable.
4. AI-enhanced governance could synchronize well with AI company regulation. These companies would be well-placed to develop innovations and could hypothetically be incentivized to do much of the work. AI-enhanced governance might be necessary to ensure that these organizations are aligned with public interests.
5. More thorough investigation here could be promising for the effective altruism community.
Within effective altruism now, there's a lot of work on governance and AI, but not much on using AI for governance. AI Governance typically focuses on using conventional strategies to oversee AI organizations, while AI Alignment research focuses on aligning AI systems. However, leveraging AI to improve human governance is an underexplored area that could complement these cause areas. You can think of it as "Organizational Alignment", as a counterpoint to "AI Alignment."
This article was written after some rough ideation I've done about this area. This isn't at all a literature review or a research agenda. That said, for those interested in this topic, here are a few posts you might find interesting.
Project ideas: Governance during explosive technological growth
The Project AI Series, by OpenMined
Safety Cases: How to Justify the Safety of Advanced AI Systems
Affirmative Safety: An Approach to Risk Management for Advanced AI
What is "AI-Assisted" Governance?
AI-Assisted Governance refers to improvements in governance that leverage artificial intelligence (AI), particularly focusing on rapidly advancing areas like Large Language Models (LLMs).
Examples methods include:
1. Monitoring politicians and executives to identify and flag misaligned or malevolent behavior, ensuring accountability and integrity.
2. Enhancing epistemics and decision-making processes at the top levels of organizations, leading to more informed and rational strategies.
3. Facilitating more effective negotiations and trades between organizations, fostering better cooperation and coordination.
4. Assisting in writing and overseeing highly secure systems, such as implementing differential privacy and formally verified, bug-free decision-automation software, for use at managerial levels.
Arguments for Governance Improvements, Generally
There's already a lot of consensus in the rationalist and effective altruist communities about the importance for governance. See the topics on
Global Governance,
AI Governance, and
Nonprofit Governance for more information.
Here are some main reasons why focusing on improving governance seems particularly promising:
Concentrated Leverage
Real-world influence is disproportionately concentrated in the hands of a relatively small number of leaders in government, business, and other pivotal institutions. This is especially true in the case of rapid AI progress. Improving the reasoning and actions of this select group is therefore perhaps the most targeted, tractable, and neglected way to shape humanity's long-term future. AI tools could offer uniquely potent levers to do so.
A lot of epistemic-enhancing work focuses on helping large populations. But some people will matter many times as much as others, and these people are often in key management positions.
Dramatic Room for Improvement
It's hard to lo...