80,000 Hours Podcast

Emergency pod: Did OpenAI give up, or is this just a new trap? (with Rose Chan Loui)

May 8, 2025

Rose Chan Loui, a nonprofit law expert at UCLA, joins the discussion about OpenAI's recent governance shift from a nonprofit to a public benefit corporation. She highlights the significance of the attorneys general's intervention and what it means for the nonprofit's control over safety decisions. Chan Loui stresses that while the words of change sound promising, their effectiveness hinges on practical enforcement and clarity. The episode also explores broader governance challenges, ethical AI development, and the potential pitfalls of corporate influence on nonprofit missions.

01:02:50

Creator website

Episode guests

Rose Chan Loui

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

OpenAI's restructuring allows the nonprofit to influence decisions as a significant shareholder, yet uncertainties remain about the actual power dynamics.

The intervention of state attorneys general highlights the tension between corporate interests and public oversight during OpenAI's transition.

Balancing profit motives with the nonprofit's commitment to safe AGI development poses challenges that require clear governance accountability measures.

Deep dives

Change in Control Structure

OpenAI's transition allows the nonprofit to maintain control while becoming a significant shareholder in the new public benefit corporation (PBC). The change indicates a shift where the nonprofit no longer has direct control but can still influence decisions through its shareholding. This restructuring raises questions about how much power the nonprofit will actually hold, particularly in ensuring the safe development of artificial general intelligence (AGI). The key takeaway is that although the nonprofit’s governance will change, its commitment to overseeing safe AI development remains a priority.

Intro

2min

Governance and Control: OpenAI's Transition

26min

Governance Challenges in Nonprofit-to-For-Profit Transitions

13min

Ensuring Board Independence at OpenAI

4min

The Ethics of AI Development

5min

Evaluating Nonprofit Control and Corporate Philanthropy

2min

Governance Challenges at OpenAI

11min

When attorneys general intervene in corporate affairs, it usually means something has gone seriously wrong. In OpenAI’s case, it appears to have forced a dramatic reversal of the company’s plans to sideline its nonprofit foundation, announced in a blog post that made headlines worldwide.

The company’s sudden announcement that its nonprofit will “retain control” credits “constructive dialogue” with the attorneys general of California and Delaware — corporate-speak for what was likely a far more consequential confrontation behind closed doors. A confrontation perhaps driven by public pressure from Nobel Prize winners, past OpenAI staff, and community organisations.

But whether this change will help depends entirely on the details of implementation — details that remain worryingly vague in the company’s announcement.

Return guest Rose Chan Loui, nonprofit law expert at UCLA, sees potential in OpenAI’s new proposal, but emphasises that “control” must be carefully defined and enforced: “The words are great, but what’s going to back that up?” Without explicitly defining the nonprofit’s authority over safety decisions, the shift could be largely cosmetic.

Links to learn more, video, and full transcript: https://80k.info/rcl4

Why have state officials taken such an interest so far? Host Rob Wiblin notes, “OpenAI was proposing that the AGs would no longer have any say over what this super momentous company might end up doing. … It was just crazy how they were suggesting that they would take all of the existing money and then pursue a completely different purpose.”

Now that they’re in the picture, the AGs have leverage to ensure the nonprofit maintains genuine control over issues of public safety as OpenAI develops increasingly powerful AI.

Rob and Rose explain three key areas where the AGs can make a huge difference to whether this plays out in the public’s best interest:

Ensuring that the contractual agreements giving the nonprofit control over the new Delaware public benefit corporation are watertight, and don’t accidentally shut the AGs out of the picture.
Insisting that a majority of board members are truly independent by prohibiting indirect as well as direct financial stakes in the business.
Insisting that the board is empowered with the money, independent staffing, and access to information which they need to do their jobs.

This episode was originally recorded on May 6, 2025.

Chapters:

Cold open (00:00:00)
Rose is back! (00:01:06)
The nonprofit will stay 'in control' (00:01:28)
Backlash to OpenAI’s original plans (00:08:22)
The new proposal (00:16:33)
Giving up the super-profits (00:20:52)
Can the nonprofit maintain control of the company? (00:24:49)
Could for profit investors sue if profits aren't prioritised? (00:33:01)
The 6 governance safeguards at risk with the restructure (00:34:33)
Will the nonprofit’s giving just be corporate PR for the for-profit? (00:49:12)
Is this good, or not? (00:51:06)
Ways this could still go wrong – but reasons for optimism (00:54:19)

Video editing: Simon Monsour and Luke Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Music: Ben Cordell
Transcriptions and web: Katy Moore