The Nonlinear Library EA - What's Going on With OpenAI's Messaging? by Ozzie Gooen
May 21, 2024
05:49
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's Going on With OpenAI's Messaging?, published by Ozzie Gooen on May 21, 2024 on The Effective Altruism Forum.
This is a quickly-written opinion piece, of what I understand about OpenAI. I first posted it to Facebook, where it had some discussion.
Some arguments that OpenAI is making, simultaneously:
1. OpenAI will likely reach and own transformative AI (useful for attracting talent to work there).
2. OpenAI cares a lot about safety (good for public PR and government regulations).
3. OpenAI isn't making anything dangerous and is unlikely to do so in the future (good for public PR and government regulations).
4. OpenAI doesn't need to spend many resources on safety, and implementing safe AI won't put it at any competitive disadvantage (important for investors who own most of the company).
5. Transformative AI will be incredibly valuable for all of humanity in the long term (for public PR and developers).
6. People at OpenAI have thought long and hard about what will happen, and it will be fine.
7. We can't predict concretely what transformative AI will look like or what will happen after (Note: Any specific scenario they propose would upset a lot of people. Vague hand-waving upsets fewer people).
8. OpenAI can be held accountable to the public because it has a capable board of advisors overseeing Sam Altman (he said this explicitly in an interview).
9. The previous board scuffle was a one-time random event that was a very minor deal.
10. OpenAI has a nonprofit structure that provides an unusual focus on public welfare.
11. The nonprofit structure of OpenAI won't inconvenience its business prospects or shareholders in any way.
12. The name "OpenAI," which clearly comes from the early days when the mission was actually to make open-source AI, is an equally good name for where the company is now.* (I don't actually care about this, but find it telling that the company doubles down on arguing the name still is applicable).
So they need to simultaneously say:
"We're making something that will dominate the global economy and outperform humans at all capabilities, including military capabilities, but is not a threat."
"Our experimental work is highly safe, but in a way that won't actually cost us anything."
"We're sure that the long-term future of transformative change will be beneficial, even though none of us can know or outline specific details of what that might actually look like."
"We have a great board of advisors that provide accountability. Sure, a few months ago, the board tried to fire Sam, and Sam was able to overpower them within two weeks, but next time will be different."
"We have all of the benefits of being a nonprofit, but we don't have any of the costs of being a nonprofit."
Meta's messaging is clearer.
"AI development won't get us to transformative AI, we don't think that AI safety will make a difference, we're just going to optimize for profitability."
Anthropic's messaging is a bit clearer
"We think that AI development is a huge deal and correspondingly scary, and we're taking a costlier approach accordingly, though not too costly such that we'd be irrelevant."
This still requires a strange and narrow worldview to make sense, but it's still more coherent.
But OpenAI's messaging has turned into a particularly tangled mess of conflicting promises. It's the kind of political strategy that can work for a while, especially if you can have most of your conversations in private, but is really hard to pull off when you're highly public and facing multiple strong competitive pressures.
If I were a journalist interviewing Sam Altman, I'd try to spend as much of it as possible just pinning him down on these countervailing promises they're making. Some types of questions I'd like him to answer would include:
"Please lay out a specific, year-by-year, story of o...
This is a quickly-written opinion piece, of what I understand about OpenAI. I first posted it to Facebook, where it had some discussion.
Some arguments that OpenAI is making, simultaneously:
1. OpenAI will likely reach and own transformative AI (useful for attracting talent to work there).
2. OpenAI cares a lot about safety (good for public PR and government regulations).
3. OpenAI isn't making anything dangerous and is unlikely to do so in the future (good for public PR and government regulations).
4. OpenAI doesn't need to spend many resources on safety, and implementing safe AI won't put it at any competitive disadvantage (important for investors who own most of the company).
5. Transformative AI will be incredibly valuable for all of humanity in the long term (for public PR and developers).
6. People at OpenAI have thought long and hard about what will happen, and it will be fine.
7. We can't predict concretely what transformative AI will look like or what will happen after (Note: Any specific scenario they propose would upset a lot of people. Vague hand-waving upsets fewer people).
8. OpenAI can be held accountable to the public because it has a capable board of advisors overseeing Sam Altman (he said this explicitly in an interview).
9. The previous board scuffle was a one-time random event that was a very minor deal.
10. OpenAI has a nonprofit structure that provides an unusual focus on public welfare.
11. The nonprofit structure of OpenAI won't inconvenience its business prospects or shareholders in any way.
12. The name "OpenAI," which clearly comes from the early days when the mission was actually to make open-source AI, is an equally good name for where the company is now.* (I don't actually care about this, but find it telling that the company doubles down on arguing the name still is applicable).
So they need to simultaneously say:
"We're making something that will dominate the global economy and outperform humans at all capabilities, including military capabilities, but is not a threat."
"Our experimental work is highly safe, but in a way that won't actually cost us anything."
"We're sure that the long-term future of transformative change will be beneficial, even though none of us can know or outline specific details of what that might actually look like."
"We have a great board of advisors that provide accountability. Sure, a few months ago, the board tried to fire Sam, and Sam was able to overpower them within two weeks, but next time will be different."
"We have all of the benefits of being a nonprofit, but we don't have any of the costs of being a nonprofit."
Meta's messaging is clearer.
"AI development won't get us to transformative AI, we don't think that AI safety will make a difference, we're just going to optimize for profitability."
Anthropic's messaging is a bit clearer
"We think that AI development is a huge deal and correspondingly scary, and we're taking a costlier approach accordingly, though not too costly such that we'd be irrelevant."
This still requires a strange and narrow worldview to make sense, but it's still more coherent.
But OpenAI's messaging has turned into a particularly tangled mess of conflicting promises. It's the kind of political strategy that can work for a while, especially if you can have most of your conversations in private, but is really hard to pull off when you're highly public and facing multiple strong competitive pressures.
If I were a journalist interviewing Sam Altman, I'd try to spend as much of it as possible just pinning him down on these countervailing promises they're making. Some types of questions I'd like him to answer would include:
"Please lay out a specific, year-by-year, story of o...