The Nonlinear Library AF - Takeoff speeds presentation at Anthropic by Tom Davidson
Jun 4, 2024
37:55
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Takeoff speeds presentation at Anthropic, published by Tom Davidson on June 4, 2024 on The AI Alignment Forum.
This is a lightly edited transcript of a presentation that I (Tom Davidson) gave about the risks of a fast takeoff at Anthropic in September 2023. See also the video recording, or the slides.
None of the content necessarily reflects the views of Anthropic or anyone who works there.
Summary:
Software progress - improvements in pre-training algorithms, data quality, prompting strategies, tooling, scaffolding, and all other sources of AI progress other than compute - has been a major driver of AI progress in recent years. I guess it's driven about half of total progress in the last 5 years.
When we have "AGI" (=AI that could fully automate AI R&D), the pace of software progress might increase dramatically (e.g. by a factor of ten).
Bottlenecks might prevent this - e.g. diminishing returns to finding software innovations, retraining new AI models from scratch, or computationally expensive experiments for finding better algorithms. But no bottleneck is decisive, and there's a real possibility that there is a period of dramatically faster capabilities progress despite all of the bottlenecks.
This period of accelerated progress might happen just when new extremely dangerous capabilities are emerging and previously-effective alignment techniques stop working.
A period of accelerated progress like this could significantly exacerbate risks from misuse, societal disruption, concentration of power, and loss of control.
To reduce these risks, labs should monitor for early warning signs of AI accelerating AI progress. In particular they can: track the pace of software progress to see if it's accelerating; run evals of whether AI systems can autonomously complete challenging AI R&D tasks; and measure the productivity gains to employees who use AI systems in their work via surveys and RCTs.
Labs should implement protective measures by the time these warning signs occur, including external oversight and info security.
Intro
Hi everyone, really great to be here. My name's Tom Davidson. I work at Open Philanthropy as a Senior Research Analyst and a lot of my work over the last couple of years has been around AI take-off speeds and the possibility that AI systems themselves could accelerate AI capabilities progress. In this talk I'm going to talk a little bit about that research, and then also about some steps that I think labs could take to reduce the risks caused by AI accelerating AI progress.
Ok, so here is the brief plan. I'm going to quickly go through some recent drivers of AI progress, which will set the scene to discuss how much AI progress might accelerate when we get AGI. Then the bulk of the talk will be focused on what risks there might be if AGI does significantly accelerate AI progress - which I think is a real possibility - and how labs can reduce those risks.
Software improvements have been a significant fraction of recent AI progress
So drivers of progress.
It's probably very familiar to many people that the compute used to train the most powerful AI models has increased very quickly over recent years. According to Epoch's accounting, about 4X increase per year.
Efficiency improvements in pre-training algorithms are a significant driver of AI progress
What I want to highlight from this slide is that algorithmic efficiency improvements - improved algorithms that allow you to train equally capable models with less compute than before - have also played a kind of comparably important role. According to Epoch's accounting, these algorithmic efficiency improvements account for more than half of the gains from compute.
That's going to be important later because when we're talking about how much the pace of progress might accelerate, today's fast algorithmic progress...
This is a lightly edited transcript of a presentation that I (Tom Davidson) gave about the risks of a fast takeoff at Anthropic in September 2023. See also the video recording, or the slides.
None of the content necessarily reflects the views of Anthropic or anyone who works there.
Summary:
Software progress - improvements in pre-training algorithms, data quality, prompting strategies, tooling, scaffolding, and all other sources of AI progress other than compute - has been a major driver of AI progress in recent years. I guess it's driven about half of total progress in the last 5 years.
When we have "AGI" (=AI that could fully automate AI R&D), the pace of software progress might increase dramatically (e.g. by a factor of ten).
Bottlenecks might prevent this - e.g. diminishing returns to finding software innovations, retraining new AI models from scratch, or computationally expensive experiments for finding better algorithms. But no bottleneck is decisive, and there's a real possibility that there is a period of dramatically faster capabilities progress despite all of the bottlenecks.
This period of accelerated progress might happen just when new extremely dangerous capabilities are emerging and previously-effective alignment techniques stop working.
A period of accelerated progress like this could significantly exacerbate risks from misuse, societal disruption, concentration of power, and loss of control.
To reduce these risks, labs should monitor for early warning signs of AI accelerating AI progress. In particular they can: track the pace of software progress to see if it's accelerating; run evals of whether AI systems can autonomously complete challenging AI R&D tasks; and measure the productivity gains to employees who use AI systems in their work via surveys and RCTs.
Labs should implement protective measures by the time these warning signs occur, including external oversight and info security.
Intro
Hi everyone, really great to be here. My name's Tom Davidson. I work at Open Philanthropy as a Senior Research Analyst and a lot of my work over the last couple of years has been around AI take-off speeds and the possibility that AI systems themselves could accelerate AI capabilities progress. In this talk I'm going to talk a little bit about that research, and then also about some steps that I think labs could take to reduce the risks caused by AI accelerating AI progress.
Ok, so here is the brief plan. I'm going to quickly go through some recent drivers of AI progress, which will set the scene to discuss how much AI progress might accelerate when we get AGI. Then the bulk of the talk will be focused on what risks there might be if AGI does significantly accelerate AI progress - which I think is a real possibility - and how labs can reduce those risks.
Software improvements have been a significant fraction of recent AI progress
So drivers of progress.
It's probably very familiar to many people that the compute used to train the most powerful AI models has increased very quickly over recent years. According to Epoch's accounting, about 4X increase per year.
Efficiency improvements in pre-training algorithms are a significant driver of AI progress
What I want to highlight from this slide is that algorithmic efficiency improvements - improved algorithms that allow you to train equally capable models with less compute than before - have also played a kind of comparably important role. According to Epoch's accounting, these algorithmic efficiency improvements account for more than half of the gains from compute.
That's going to be important later because when we're talking about how much the pace of progress might accelerate, today's fast algorithmic progress...