OpenAI Nonprofit Buyout: Much More Than You Wanted To Know

Last month, I put out a request for experts to help me understand the details of OpenAI’s forprofit buyout. The following comes from someone who has looked into the situation in depth but is not an insider. Mistakes are mine alone.

Why Was OpenAI A Nonprofit In The First Place?

In the early 2010s, the AI companies hadn’t yet discovered scaling laws, and so underestimated the amount of compute (and therefore money) it would take to build AI. DeepMind was the first victim; originally founded on high ideals of prioritizing safety and responsible stewardship of the Singularity, it hit a financial barrier and sold to Google.

This scared Elon Musk, who didn’t trust Google (or any corporate sponsor) with AGI. He teamed up with Sam Altman and others, and OpenAI was born. To avoid duplicating DeepMind’s failure, they founded it as a nonprofit with a mission to “build safe and beneficial artificial general intelligence for the benefit of humanity”.

But like DeepMind, OpenAI needed money. At first, they scraped by with personal donations from Musk and other idealists, but as the full impact of scaling laws became clearer, Altman wanted to form a forprofit arm and seek investment. Musk and Altman disagree on what happened next: Musk said he objected to the profit focus, Altman says Musk agreed but wanted to be in charge. In any case, Musk left, Altman took full control, and OpenAI founded a forprofit subsidiary.

This subsidary was supposedly a “capped forprofit”, meaning that their investors were capped at 100x return - if someone invested $1 million, they could get a max of $100 million back, no matter how big OpenAI became - this ensured that the majority of gains from a Singularity would go to humanity rather than investors. But a capped forprofit isn’t a real kind of corporate structure; in real life OpenAI handles this through Profit Participation Units, a sort of weird stock/bond hybrid which does what OpenAI claims the capped forprofit model is doing.

https://www.astralcodexten.com/p/openai-nonprofit-buyout-much-more