The Nonlinear Library EA - titotal on AI risk scepticism by Vasco Grilo
May 31, 2024
10:30
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: titotal on AI risk scepticism, published by Vasco Grilo on May 31, 2024 on The Effective Altruism Forum.
This is a linkpost for titotal's posts on AI risk scepticism, which I think are great. I list the posts below chronologically.
Chaining the evil genie: why "outer" AI safety is probably easy
Conclusion
Summing up my argument in TLDR format:
1. For each AGI, there will be tasks that have difficulty beyond it's capabilities.
2. You can make the task "subjugate humanity under these constraints" arbitrarily more difficult or undesirable by adding more and more constraints to a goal function.
3. A lot of these constraints are quite simple, but drastically effective, such as implementing time limits, bounded goals, and prohibitions on human death.
4. Therefore, it is not very difficult to design a useful goal function that raises subjugation difficulty above the capability level of the AGI, simply by adding arbitrarily many constraints.
Even if you disagree with some of these points, it seems hard to see how a constrained AI wouldn't at least have a greatly reduced probability of successful subjugation, so I think it makes sense to pursue constraints anyway (as I'm sure plenty of people already are).
AGI Battle Royale: Why "slow takeover" scenarios devolve into a chaotic multi-AGI fight to the death
Summary
The main argument goes as follows:
1. Malevolent AGI's (in the standard model of unbounded goal maximisers) will almost all have incompatible end goals, making each AGI is an existential threat to every other AGI.
2. Once one AGI exists, others are likely not far behind, possibly at an accelerating rate.
3. Therefore, if early AGI can't take over immediately, there will be a complex, chaotic shadow war between multiple AGI's with the ultimate aim of destroying every other AI and humanity.
I outlined a few scenarios of how this might play out, depending on what assumptions you make:
Scenario a: Fast-ish takeoff
The AGI is improving fast enough that it can tolerate a few extra enemies. It boosts itself until the improvement saturates, takes a shot at humanity, and then dukes it out with other AGI after we are dead.
Scenario b: Kamikaze scenario
The AGI can't improve fast enough to keep up with new AGI generation. It attacks immediately, no matter how slim the odds, because it is doomed either way.
Scenario c: AGI induced slowdown
The AGI figures out a way to quickly sabotage the growth of new AGI's, allowing it to outpace their growth and switch to scenario a.
Scenario d: AI cooperation
Different AGI's work together and pool power to defeat humanity cooperatively, then fight each other afterwards.
Scenario e: Crabs in a bucket
Different AGI's constantly tear down whichever AI is "winning", so the AI are too busy fighting each other to ever take us down.
I hope people find this analysis interesting! I doubt I'm the first person to think of these points, but I thought it was worth giving an independent look at it.
How "AGI" could end up being many different specialized AI's stitched together
Summary:
In this post, I am arguing that advanced AI may consist of many different smaller AI modules stitched together in a modular fashion. The argument goes as follows:
1. Existing AI is already modular in nature, in that it is wrapped into larger, modular, "dumb" code.
2. In the near-term, you can produce far more impressive results by stitching together different specialized AI modules than by trying to force one AI to do everything.
3. This trend could continue into the future, as specialized AI can have their architecture, goals and data can be customized for maximum performance in each specific sub-field.
I then explore a few implications this type of AI system might have for AI safety, concluding that it might result in disunified or idiot savant AI's (helping humanity), or ...
This is a linkpost for titotal's posts on AI risk scepticism, which I think are great. I list the posts below chronologically.
Chaining the evil genie: why "outer" AI safety is probably easy
Conclusion
Summing up my argument in TLDR format:
1. For each AGI, there will be tasks that have difficulty beyond it's capabilities.
2. You can make the task "subjugate humanity under these constraints" arbitrarily more difficult or undesirable by adding more and more constraints to a goal function.
3. A lot of these constraints are quite simple, but drastically effective, such as implementing time limits, bounded goals, and prohibitions on human death.
4. Therefore, it is not very difficult to design a useful goal function that raises subjugation difficulty above the capability level of the AGI, simply by adding arbitrarily many constraints.
Even if you disagree with some of these points, it seems hard to see how a constrained AI wouldn't at least have a greatly reduced probability of successful subjugation, so I think it makes sense to pursue constraints anyway (as I'm sure plenty of people already are).
AGI Battle Royale: Why "slow takeover" scenarios devolve into a chaotic multi-AGI fight to the death
Summary
The main argument goes as follows:
1. Malevolent AGI's (in the standard model of unbounded goal maximisers) will almost all have incompatible end goals, making each AGI is an existential threat to every other AGI.
2. Once one AGI exists, others are likely not far behind, possibly at an accelerating rate.
3. Therefore, if early AGI can't take over immediately, there will be a complex, chaotic shadow war between multiple AGI's with the ultimate aim of destroying every other AI and humanity.
I outlined a few scenarios of how this might play out, depending on what assumptions you make:
Scenario a: Fast-ish takeoff
The AGI is improving fast enough that it can tolerate a few extra enemies. It boosts itself until the improvement saturates, takes a shot at humanity, and then dukes it out with other AGI after we are dead.
Scenario b: Kamikaze scenario
The AGI can't improve fast enough to keep up with new AGI generation. It attacks immediately, no matter how slim the odds, because it is doomed either way.
Scenario c: AGI induced slowdown
The AGI figures out a way to quickly sabotage the growth of new AGI's, allowing it to outpace their growth and switch to scenario a.
Scenario d: AI cooperation
Different AGI's work together and pool power to defeat humanity cooperatively, then fight each other afterwards.
Scenario e: Crabs in a bucket
Different AGI's constantly tear down whichever AI is "winning", so the AI are too busy fighting each other to ever take us down.
I hope people find this analysis interesting! I doubt I'm the first person to think of these points, but I thought it was worth giving an independent look at it.
How "AGI" could end up being many different specialized AI's stitched together
Summary:
In this post, I am arguing that advanced AI may consist of many different smaller AI modules stitched together in a modular fashion. The argument goes as follows:
1. Existing AI is already modular in nature, in that it is wrapped into larger, modular, "dumb" code.
2. In the near-term, you can produce far more impressive results by stitching together different specialized AI modules than by trying to force one AI to do everything.
3. This trend could continue into the future, as specialized AI can have their architecture, goals and data can be customized for maximum performance in each specific sub-field.
I then explore a few implications this type of AI system might have for AI safety, concluding that it might result in disunified or idiot savant AI's (helping humanity), or ...