The Nonlinear Library: LessWrong LW - OpenAI: Exodus by Zvi
May 20, 2024
01:09:13
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: OpenAI: Exodus, published by Zvi on May 20, 2024 on LessWrong.
Previously: OpenAI: Facts From a Weekend, OpenAI: The Battle of the Board, OpenAI: Leaks Confirm the Story, OpenAI: Altman Returns, OpenAI: The Board Expands.
Ilya Sutskever and Jan Leike have left OpenAI. This is almost exactly six months after Altman's temporary firing and The Battle of the Board, the day after the release of GPT-4o, and soon after a number of other recent safety-related OpenAI departures. Many others working on safety have also left recently. This is part of a longstanding pattern at OpenAI.
Jan Leike later offered an explanation for his decision on Twitter. Leike asserts that OpenAI has lost the mission on safety and culturally been increasingly hostile to it. He says the superalignment team was starved for resources, with its public explicit compute commitments dishonored, and that safety has been neglected on a widespread basis, not only superalignment but also including addressing the safety needs of the GPT-5 generation of models.
Altman acknowledged there was much work to do on the safety front. Altman and Brockman then offered a longer response that seemed to say exactly nothing new.
Then we learned that OpenAI has systematically misled and then threatened its departing employees, forcing them to sign draconian lifetime non-disparagement agreements, which they are forbidden to reveal due to their NDA.
Altman has to some extent acknowledged this and promised to fix it once the allegations became well known, but so far there has been no fix implemented beyond an offer to contact him privately for relief.
These events all seem highly related.
Also these events seem quite bad.
What is going on?
This post walks through recent events and informed reactions to them.
The first ten sections address departures from OpenAI, especially Sutskever and Leike.
The next five sections address the NDAs and non-disparagement agreements.
Then at the end I offer my perspective, highlight another, and look to paths forward.
Table of Contents
1. The Two Departure Announcements
2. Who Else Has Left Recently?
3. Who Else Has Left Overall?
4. Early Reactions to the Departures
5. The Obvious Explanation: Altman
6. Jan Leike Speaks
7. Reactions After Lekie's Statement
8. Greg Brockman and Sam Altman Respond to Leike
9. Reactions from Some Folks Unworried About Highly Capable AI
10. Don't Worry, Be Happy?
11. The Non-Disparagement and NDA Clauses
12. Legality in Practice
13. Implications and Reference Classes
14. Altman Responds on Non-Disparagement Clauses
15. So, About That Response
16. How Bad Is All This?
17. Those Who Are Against These Efforts to Prevent AI From Killing Everyone
18. What Will Happen Now?
19. What Else Might Happen or Needs to Happen Now?
The Two Departure Announcements
Here are the full announcements and top-level internal statements made on Twitter around the departures of Ilya Sutskever and Jan Leike.
Ilya Sutskever: After almost a decade, I have made the decision to leave OpenAI. The company's trajectory has been nothing short of miraculous, and I'm confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of Jakub Pachocki. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything.
I am excited for what comes next - a project that is very personally meaningful to me about which I will share details in due time.
[Ilya then shared the photo below]
Jakub Pachocki: Ilya introduced me to the world of deep learning research, and has been a mentor to me, and a great collaborator for many years. His incredible vision for what deep learning could become was foundational to what OpenAI, and the field of AI, is today. I...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: OpenAI: Exodus, published by Zvi on May 20, 2024 on LessWrong.
Previously: OpenAI: Facts From a Weekend, OpenAI: The Battle of the Board, OpenAI: Leaks Confirm the Story, OpenAI: Altman Returns, OpenAI: The Board Expands.
Ilya Sutskever and Jan Leike have left OpenAI. This is almost exactly six months after Altman's temporary firing and The Battle of the Board, the day after the release of GPT-4o, and soon after a number of other recent safety-related OpenAI departures. Many others working on safety have also left recently. This is part of a longstanding pattern at OpenAI.
Jan Leike later offered an explanation for his decision on Twitter. Leike asserts that OpenAI has lost the mission on safety and culturally been increasingly hostile to it. He says the superalignment team was starved for resources, with its public explicit compute commitments dishonored, and that safety has been neglected on a widespread basis, not only superalignment but also including addressing the safety needs of the GPT-5 generation of models.
Altman acknowledged there was much work to do on the safety front. Altman and Brockman then offered a longer response that seemed to say exactly nothing new.
Then we learned that OpenAI has systematically misled and then threatened its departing employees, forcing them to sign draconian lifetime non-disparagement agreements, which they are forbidden to reveal due to their NDA.
Altman has to some extent acknowledged this and promised to fix it once the allegations became well known, but so far there has been no fix implemented beyond an offer to contact him privately for relief.
These events all seem highly related.
Also these events seem quite bad.
What is going on?
This post walks through recent events and informed reactions to them.
The first ten sections address departures from OpenAI, especially Sutskever and Leike.
The next five sections address the NDAs and non-disparagement agreements.
Then at the end I offer my perspective, highlight another, and look to paths forward.
Table of Contents
1. The Two Departure Announcements
2. Who Else Has Left Recently?
3. Who Else Has Left Overall?
4. Early Reactions to the Departures
5. The Obvious Explanation: Altman
6. Jan Leike Speaks
7. Reactions After Lekie's Statement
8. Greg Brockman and Sam Altman Respond to Leike
9. Reactions from Some Folks Unworried About Highly Capable AI
10. Don't Worry, Be Happy?
11. The Non-Disparagement and NDA Clauses
12. Legality in Practice
13. Implications and Reference Classes
14. Altman Responds on Non-Disparagement Clauses
15. So, About That Response
16. How Bad Is All This?
17. Those Who Are Against These Efforts to Prevent AI From Killing Everyone
18. What Will Happen Now?
19. What Else Might Happen or Needs to Happen Now?
The Two Departure Announcements
Here are the full announcements and top-level internal statements made on Twitter around the departures of Ilya Sutskever and Jan Leike.
Ilya Sutskever: After almost a decade, I have made the decision to leave OpenAI. The company's trajectory has been nothing short of miraculous, and I'm confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of Jakub Pachocki. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything.
I am excited for what comes next - a project that is very personally meaningful to me about which I will share details in due time.
[Ilya then shared the photo below]
Jakub Pachocki: Ilya introduced me to the world of deep learning research, and has been a mentor to me, and a great collaborator for many years. His incredible vision for what deep learning could become was foundational to what OpenAI, and the field of AI, is today. I...