The Nonlinear Library LW - Twitter thread on politics of AI safety by Richard Ngo
Jul 31, 2024
02:24
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Twitter thread on politics of AI safety, published by Richard Ngo on July 31, 2024 on LessWrong.
Some thoughts about the politics of AI safety, copied over (with slight modifications) from my recent twitter thread:
Risks that seem speculative today will become common sense as AI advances. The pros and cons of different safety strategies will also become much clearer over time. So our main job now is to empower future common-sense decision-making. Understanding model cognition and behavior is crucial for making good decisions. But equally important is ensuring that key institutions are able to actually process that knowledge.
Institutions can lock in arbitrarily crazy beliefs via preference falsification. When someone contradicts the party line, even people who agree face pressure to condemn them. We saw this with the Democrats hiding evidence of Biden's mental decline. It's also a key reason why dictators can retain power even after almost nobody truly supports them.
I worry that DC has already locked in an anti-China stance, which could persist even if most individuals change their minds. We're also trending towards Dems and Republicans polarizing on the safety/accelerationism axis. This polarization is hard to fight directly. But there will be an increasing number of "holy shit" moments that serve as Schelling points to break existing consensus. It will be very high-leverage to have common-sense bipartisan frameworks and proposals ready for those moments.
Perhaps the most crucial desideratum for these proposals is that they're robust to the inevitable scramble for power that will follow those "holy shit" movements. I don't know how to achieve that, but one important factor is: will AI tools and assistants help or hurt? E.g. truth-motivated AI could help break preference falsification. But conversely, centralized control of AIs used in govts could make it easier to maintain a single narrative.
This problem of "governance with AI" (as opposed to governance *of* AI) seems very important! Designing principles for integrating AI into human governments feels analogous in historical scope to writing the US constitution. One bottleneck in making progress on that: few insiders disclose how NatSec decisions are really made (though Daniel Ellsberg's books are a notable exception). So I expect that understanding this better will be a big focus of mine going forward.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
Some thoughts about the politics of AI safety, copied over (with slight modifications) from my recent twitter thread:
Risks that seem speculative today will become common sense as AI advances. The pros and cons of different safety strategies will also become much clearer over time. So our main job now is to empower future common-sense decision-making. Understanding model cognition and behavior is crucial for making good decisions. But equally important is ensuring that key institutions are able to actually process that knowledge.
Institutions can lock in arbitrarily crazy beliefs via preference falsification. When someone contradicts the party line, even people who agree face pressure to condemn them. We saw this with the Democrats hiding evidence of Biden's mental decline. It's also a key reason why dictators can retain power even after almost nobody truly supports them.
I worry that DC has already locked in an anti-China stance, which could persist even if most individuals change their minds. We're also trending towards Dems and Republicans polarizing on the safety/accelerationism axis. This polarization is hard to fight directly. But there will be an increasing number of "holy shit" moments that serve as Schelling points to break existing consensus. It will be very high-leverage to have common-sense bipartisan frameworks and proposals ready for those moments.
Perhaps the most crucial desideratum for these proposals is that they're robust to the inevitable scramble for power that will follow those "holy shit" movements. I don't know how to achieve that, but one important factor is: will AI tools and assistants help or hurt? E.g. truth-motivated AI could help break preference falsification. But conversely, centralized control of AIs used in govts could make it easier to maintain a single narrative.
This problem of "governance with AI" (as opposed to governance *of* AI) seems very important! Designing principles for integrating AI into human governments feels analogous in historical scope to writing the US constitution. One bottleneck in making progress on that: few insiders disclose how NatSec decisions are really made (though Daniel Ellsberg's books are a notable exception). So I expect that understanding this better will be a big focus of mine going forward.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org