The Nonlinear Library EA - Artiabout recent OpenAI departures by bruce
May 17, 2024
01:55
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Artiabout recent OpenAI departures, published by bruce on May 17, 2024 on The Effective Altruism Forum.
A brief overview of recent OpenAI departures (Ilya Sutskever, Jan Leike, Daniel Kokotajlo, Leopold Aschenbrenner, Pavel Izmailov, William Saunders, Ryan Lowe Cullen O'Keefe[1]). Will add other relevant media pieces below as I come across them.
Some quotes perhaps worth highlighting:
Even when the team was functioning at full capacity, that "dedicated investment" was home to a tiny fraction of OpenAI's researchers and was promised only 20 percent of its computing power - perhaps the most important resource at an AI company. Now, that computing power may be siphoned off to other OpenAI teams, and it's unclear if there'll be much focus on avoiding catastrophic risk from future AI models.
Jan suggesting that compute for safety may have been deprioritised even despite the 20% commitment. (Wired claims that OpenAI confirms that their "superalignment team is no more").
"I joined with substantial hope that OpenAI would rise to the occasion and behave more responsibly as they got closer to achieving AGI. It slowly became clear to many of us that this would not happen," Kokotajlo told me. "I gradually lost trust in OpenAI leadership and their ability to responsibly handle AGI, so I quit."
(Additional kudos to Daniel Kokotajlo for not signing additional confidentiality obligations on departure, which is plausibly relevant for Jan too given his recent thread).
Edit:
Shakeel's article on the same topic.
Kelsey's article about the nondisclosure/nondisparagement provisions that OpenAI employees have been offered
Wired claims that OpenAI confirms that their "superalignment team is no more".
1. ^
Covered by Shakeel/Wired, but thought it'd be clearer to list all names together
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
A brief overview of recent OpenAI departures (Ilya Sutskever, Jan Leike, Daniel Kokotajlo, Leopold Aschenbrenner, Pavel Izmailov, William Saunders, Ryan Lowe Cullen O'Keefe[1]). Will add other relevant media pieces below as I come across them.
Some quotes perhaps worth highlighting:
Even when the team was functioning at full capacity, that "dedicated investment" was home to a tiny fraction of OpenAI's researchers and was promised only 20 percent of its computing power - perhaps the most important resource at an AI company. Now, that computing power may be siphoned off to other OpenAI teams, and it's unclear if there'll be much focus on avoiding catastrophic risk from future AI models.
Jan suggesting that compute for safety may have been deprioritised even despite the 20% commitment. (Wired claims that OpenAI confirms that their "superalignment team is no more").
"I joined with substantial hope that OpenAI would rise to the occasion and behave more responsibly as they got closer to achieving AGI. It slowly became clear to many of us that this would not happen," Kokotajlo told me. "I gradually lost trust in OpenAI leadership and their ability to responsibly handle AGI, so I quit."
(Additional kudos to Daniel Kokotajlo for not signing additional confidentiality obligations on departure, which is plausibly relevant for Jan too given his recent thread).
Edit:
Shakeel's article on the same topic.
Kelsey's article about the nondisclosure/nondisparagement provisions that OpenAI employees have been offered
Wired claims that OpenAI confirms that their "superalignment team is no more".
1. ^
Covered by Shakeel/Wired, but thought it'd be clearer to list all names together
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org