The Nonlinear Library: LessWrong LW - MIRI's June 2024 Newsletter by Harlan
Jun 15, 2024
04:35
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MIRI's June 2024 Newsletter, published by Harlan on June 15, 2024 on LessWrong.
MIRI updates
MIRI Communications Manager Gretta Duleba explains
MIRI's current communications strategy. We hope to clearly communicate to policymakers and the general public why there's an urgent need to shut down frontier AI development, and make the case for installing an "off-switch". This will not be easy, and there is a lot of work to be done. Some projects we're currently exploring include a new website, a book, and an online reference resource.
Rob Bensinger
argues, contra Leopold Aschenbrenner, that the US government should not race to develop artificial superintelligence. "If anyone builds it, everyone dies." Instead, Rob outlines a proposal for the US to spearhead an international alliance to halt progress toward the technology.
At the end of June, the
Agent Foundations team, including Scott Garrabrant and others, will be parting ways with MIRI to continue their work as independent researchers. The team was originally set up and "sponsored" by Nate Soares and Eliezer Yudkowsky. However, as AI capabilities have progressed rapidly in recent years, Nate and Eliezer have become increasingly pessimistic about this type of work yielding significant results within the relevant timeframes.
Consequently, they have shifted their focus to other priorities.
Senior MIRI leadership explored various alternatives, including reorienting the Agent Foundations team's focus and transitioning them to an independent group under MIRI fiscal sponsorship with restricted funding, similar to
AI Impacts. Ultimately, however, we decided that parting ways made the most sense.
The Agent Foundations team has produced some stellar work over the years, and made a true attempt to tackle one of the most crucial challenges humanity faces today. We are deeply grateful for their many years of service and collaboration at MIRI, and we wish them the very best in their future endeavors.
The Technical Governance Team
responded to NIST's
request for comments on draft documents related to the
AI Risk Management Framework. The team also sent
comments in response to the "
Framework for MItigating AI Risks" put forward by U.S. Senators Mitt Romney (R-UT), Jack Reed (D-RI), Jerry Moran (R-KS), and Angus King (I-ME).
Brittany Ferrero has joined MIRI's operations team. Previously, she worked on projects such as the
Embassy Network and
Open Lunar Foundation. We're excited to have her help to execute on our mission.
News and links
AI alignment researcher Paul Christiano was
appointed as head of AI safety at the US AI Safety Institute. Last fall, Christiano
published some of his thoughts about AI regulation as well as responsible scaling policies.
The
Superalignment team at OpenAI has been disbanded following the departure of its co-leaders Ilya Sutskever and Jan Leike. The team was
launched last year to try to solve the AI alignment problem in four years. However, Leike says that
the team struggled to get the compute it needed and that "safety culture and processes have taken a backseat to shiny products" at OpenAI. This seems extremely concerning from the perspective of evaluating OpenAI's seriousness when it comes to safety and robustness work, particularly given that a similar OpenAI exodus occurred in 2020 in the wake of concerns about OpenAI's commitment to solving the alignment problem.
Vox's Kelsey Piper reports that employees who left
OpenAI were subject to an extremely restrictive NDA indefinitely preventing them from criticizing the company (or admitting that they were under an NDA), under threat of losing their vested equity in the company. OpenAI executives have since contacted former employees to say that
they will not enforce the NDAs. Rob Bensinger comments on these developments
here, strongly criticizing OpenAI for...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MIRI's June 2024 Newsletter, published by Harlan on June 15, 2024 on LessWrong.
MIRI updates
MIRI Communications Manager Gretta Duleba explains
MIRI's current communications strategy. We hope to clearly communicate to policymakers and the general public why there's an urgent need to shut down frontier AI development, and make the case for installing an "off-switch". This will not be easy, and there is a lot of work to be done. Some projects we're currently exploring include a new website, a book, and an online reference resource.
Rob Bensinger
argues, contra Leopold Aschenbrenner, that the US government should not race to develop artificial superintelligence. "If anyone builds it, everyone dies." Instead, Rob outlines a proposal for the US to spearhead an international alliance to halt progress toward the technology.
At the end of June, the
Agent Foundations team, including Scott Garrabrant and others, will be parting ways with MIRI to continue their work as independent researchers. The team was originally set up and "sponsored" by Nate Soares and Eliezer Yudkowsky. However, as AI capabilities have progressed rapidly in recent years, Nate and Eliezer have become increasingly pessimistic about this type of work yielding significant results within the relevant timeframes.
Consequently, they have shifted their focus to other priorities.
Senior MIRI leadership explored various alternatives, including reorienting the Agent Foundations team's focus and transitioning them to an independent group under MIRI fiscal sponsorship with restricted funding, similar to
AI Impacts. Ultimately, however, we decided that parting ways made the most sense.
The Agent Foundations team has produced some stellar work over the years, and made a true attempt to tackle one of the most crucial challenges humanity faces today. We are deeply grateful for their many years of service and collaboration at MIRI, and we wish them the very best in their future endeavors.
The Technical Governance Team
responded to NIST's
request for comments on draft documents related to the
AI Risk Management Framework. The team also sent
comments in response to the "
Framework for MItigating AI Risks" put forward by U.S. Senators Mitt Romney (R-UT), Jack Reed (D-RI), Jerry Moran (R-KS), and Angus King (I-ME).
Brittany Ferrero has joined MIRI's operations team. Previously, she worked on projects such as the
Embassy Network and
Open Lunar Foundation. We're excited to have her help to execute on our mission.
News and links
AI alignment researcher Paul Christiano was
appointed as head of AI safety at the US AI Safety Institute. Last fall, Christiano
published some of his thoughts about AI regulation as well as responsible scaling policies.
The
Superalignment team at OpenAI has been disbanded following the departure of its co-leaders Ilya Sutskever and Jan Leike. The team was
launched last year to try to solve the AI alignment problem in four years. However, Leike says that
the team struggled to get the compute it needed and that "safety culture and processes have taken a backseat to shiny products" at OpenAI. This seems extremely concerning from the perspective of evaluating OpenAI's seriousness when it comes to safety and robustness work, particularly given that a similar OpenAI exodus occurred in 2020 in the wake of concerns about OpenAI's commitment to solving the alignment problem.
Vox's Kelsey Piper reports that employees who left
OpenAI were subject to an extremely restrictive NDA indefinitely preventing them from criticizing the company (or admitting that they were under an NDA), under threat of losing their vested equity in the company. OpenAI executives have since contacted former employees to say that
they will not enforce the NDAs. Rob Bensinger comments on these developments
here, strongly criticizing OpenAI for...