Speaker 2
Yeah, that sounds right. Okay, so let's talk about another argument against explosive growth. And that has to do with R&D specifically. So you'll remember that at the beginning of this conversation, we talked about a reason for growth, which is that you get these increasing returns to scale with the economy from automating R&D. But maybe it just turns out that the ideas just get really hard to find really quickly, that you actually don't get much juice out of being able to automate R&D because you just run out of like really economically valuable new technologies, new ideas. So does that seem like enough
Speaker 1
to prevent this kind of rapid growth and does that seem likely? This could be enough to prevent economic growth. Now it needs to be the case that it's not just, it gets like much much harder at a very steep, steep rate, but it needs to basically get almost impossible to be able to improve the efficiency of your systems, as long as you have basically constant returns to scale in the non-idea inputs. increasing returns is you need basically constant returns in the physical inputs and then at least some oomph in terms of returns to R&D. Now, of course, the argument isn't very strong if it's just over the edge. And so I think returns to R&D being very low would make me much less convinced of this increasing returns of skill story. But the estimates that we have for the returns to R&D are not basically such that eventually R&D becomes impossible. that we might need to increase the inputs by larger and larger factors. And this factor increases by maybe 30% per doubling of technological capability or something like that, which is not great. But I think this is well within the kind of parameter values that would give rise to explosive growth.
Speaker 2
Okay, and I guess a related thought is that R&D on AI specifically might just get really hard. That is to say, it might be really, really prohibitively expensive and difficult to get capabilities good enough so that these arguments about AI automating R&D and automating loads of human jobs kicks in. How might that happen? How might it just become suddenly very difficult to keep getting these historical returns to AI R&D? So yeah, maybe in AI specifically, like it might turn
Speaker 1
out that, you know, historically in the field of AI, we have hit these types of AI winters. And, and I think that lends some credibility to this idea that we might end up, you know, again, having this winter that maybe lasts for some time, where not very much research happens or the research that happens doesn't enable us to do more with less. I think that there's no particular reason to expect this might happen. I mean, maybe you think that the entire paradigm of deep learning is doomed or something. And so some people might expect this. I don't personally expect this. Another thing to keep in mind that even just with basically current algorithms, given what we understand about scaling of AI systems, it might just be sufficient for us to scale up the amount of compute used in training to get very capable systems without doing much of R&D and discovering novel algorithms. Maybe just our industrial capacity, if it's sufficiently expanded, itself is enough to get the systems that are able to substitute very widely for human workers. And then you might ask, well, how much does our industrial capacity need to expand to produce enough chips to? You know really get us there and to also produce enough power and energy infrastructure and things like this and you know There's some reports about that try to estimate, you know Some bounds on how much compute you might need in order to train a system that ends up being as capable as a human on a broad range of tasks. I won't go into the details there, but it seems like, you know, on the order of 10 orders of magnitude, so 10 to the 10 of a factor of additional compute is needed. And so, you know, this is a very large factor. Now, it is possible that if we can continue improving the efficiency of hardware and scaling up the amount of computation that we do, that we get pretty close to that. And once we get pretty close, I think maybe that enables us to do some automation of R&D and kickstart that process to get these very capable systems. Yeah,
Speaker 2
I think I was actually mixing up in my head AI R&D and just scaling up the inputs to training. And so I guess, you know, historically, these things happen together, but we are spending more and more in terms of computing money on training runs. And we're also coming up with new ideas for how to train and run AI systems more efficiently, right? Like new algorithms. And I guess you as an epoch has some estimates for the kind of rate of progress in both these things. But you're saying something like even if algorithmic progress slows down significantly for some reason, so you see a kind of research winter, then we can still scale up inputs by like 10 or 100x or maybe more. And that might be enough to start getting AI to do some of the R&D for you. But then secondly, it's like why should you expect a winter anytime soon given that the kind of progress and algorithms has just been like pretty impressive for the last few years. Okay, so I guess since we're talking about R&D in general, we might also talk about R&D on AI specifically, right? So obviously, AI needs to get a bunch better before these arguments for explosive growth kick in a talk, right? Before it's substituting for all kinds of human work, before it's doing like meaningful amounts of R&D itself. But maybe you might just think that we're not going to get to that point anytime soon because it just gets really, really hard and expensive to build much more capable AI. One reason for that might be that the returns to R&D on AI itself slow down pretty quickly. Are there any reasons to expect that? I basically think the
Speaker 1
answer is no. So I have some work looking at algorithmic progress in language models and computer vision models. There is this broader literature on the kind of rate of algorithmic progress in software. And one striking thing is that it seems to be the case that within machine learning or within AI, specifically the rate of algorithmic progress is kind of faster than the rate that we see for other domains of software. So if we compare it to SAT solvers or linear integer programs or other things or, you know, chess engines. And so we've seen just really great returns to R&D in AI in the last, you know, 10 years. It seems kind of unlikely for us to go from this regime where the returns are really quite great to a regime where returns are really terrible. I think that itself strikes me as being not likely anytime soon. So, you know, in our language model paper we actually tried to look at has this change over time. And the answer there is if anything it might have even gotten faster. Now, you know, that is consistent with just kind of the same rate the very least, the returns look really quite great. I think the other thing is that even if the returns are much lower, it becomes really hard because we kind of hit the limits of what deep learning can do, there's still this ability to scale along the margin of expanding more compute on training. And there we can just rely on our industrial capacity to just channel more resources into building data centers, lithography machines, and fabs, channeling more of those fabs to producing AI-relevant chips. And that could give you many orders of magnitude of additional scale-up. On top of that, we've had 70 years of progress in hardware. This is Moore's law. And so although there's lots of, there's lots of speculations about how that is coming to an end, and there's specific forms which have likely ended, there's still some, you know, rate of improvement that you see on just the hardware front. So we might get the slowdown in hardware, we even get a slowdown in software, though I don't think it's particularly likely. We still have this industrial capacity that we can expand. Now, of course, if it becomes a lot harder to make progress on any one of these dimensions, the expansion, the expansions along these dimensions should be correlated. Like if it turns out that AI just doesn't work very well, then we're not going to be as eager to expand on these other dimensions.
Speaker 2
Yeah, like I've also heard that a lot of algorithmic progress comes from being able to run experiments at larger scales, and then just like learning new things, which you probably couldn't have learned from the smaller training runs. And so you might think that in particular these kind of the input of just like how big is your training run is going to be correlated with your rate of progress
Speaker 1
on finding new algorithmic efficiency insights. it's quite interesting. The rate of software progress has been roughly the same as the rate of hardware progress, which is kind of this mysterious coincidence. And I think one explanation is just that scaling up your hardware enables you to do types of experiments that unlock new kind of software innovations. And the fact that we have quite a lot of scaling we can do on just expanding our industrial capacity dedicated to compute means that we could access many orders of magnitude larger scales and with that gain kind of algorithmic insights. Can you just specifically say what we know about the rates of progress in
Speaker 2
algorithmic efficiency and what that means?
Speaker 1
Sure. So I mentioned before that there is this coincidence in software progress with the rate of hardware progress. So that's like a doubling of efficiency every roughly two years in many domains of software. In AI specifically, we see faster reads of progress. We see on the order of a doubling of efficiency or halving of the amount of compute you need every 8 to 12 months in language models, in computer vision systems, in reinforcement learning systems. This is consistent with some kind of observations of people actually doing training runs, doing these types of time travel experiments where they think, okay, if I only had the access to algorithms from like 2012 and I tried to implement them today versus if I have algorithms of today I implemented on kind of of 10 years ago, those are types of experiments. In AI, I also suggest that there is this growth in efficiency doubling every year, roughly.
Speaker 2
Yeah, gotcha. So I guess the bottom line is that just right up to the present day, we're observing a bunch of trends. So one is this kind of exponential, if somewhat slower than the other trends, growth in like price performance in hardware, which is just Moore's law, there is this much faster trend of algorithmic progress, and then also inputs are increasing. And so it seems like more than one of those trends would need to slow down for it not to be the case that AI runs into the kind of capability zone where these arguments for rapid growth are kicking in. Is that the thought? Yeah, that's right. Or
Speaker 1
it does so over the course of multiple, you know, a century or something like that, where like, you know, we kind of kind of get close, but we kind of run out of steam and just very slowly move along and have a new innovation every decade or so, such that it takes just extremely long, which spreads out the kind of effects of AI automation on our output, such that our growth rates don't noticeably accelerate.
Speaker 2
In terms of specific limits to continued progress in AI, some people might point to running into a data wall, for instance, as just one of these hard limits on scaling deep learning. Does that particular bottleneck seem like it could really throw a spanner in the works? Yeah,
Speaker 1
I think there's a question of where is this data wall? And there's a question of how much scaling does this, does the existing stock of like human generated data enable you to do in an efficient way? And then there's a question of, okay, given that we hit this data wall, is there something that we could do about alleviating this constraint? And so onto the first of these two questions, it seems just to be the case that we can do quite a lot of scaling with the existing stock of human-generated internet data. We're not quite close to exhausting the existing stock. We can do maybe a training run that outscales GPT-4 by the same factor that GPT-4 outscales GPT-2. So a thousand-fold, roughly, of additional scaling is permitted by the existing stock of data, which if you compare the rudimentary reasoning abilities of GPT-2 to the pretty sophisticated problem-solving abilities of GPT-4, suggests that you could get like pretty like a large jump in capabilities with just the existing stock. And then on top of that, there are, you know, these arguments that you can find substitutes. And one substitute that people have pointed to in particular is synthetic data where this is generated
Speaker 2
by AI. Okay, that all makes sense. Yeah, another comment that you might make to all these arguments for explosive growth is that we're talking in terms of these measures of growth, right, in terms of, you know, gross output. And among other things, introducing all these kind of new AI tools might introduce a bunch of like new products and just change the world in all sorts of ways. Such as these measures just kind of break and they're no longer tracking stuff we think is important. I don't even really know if that is an argument against explosive growth happening so much as an argument against it being measured properly, but I don't know if you have anything to say to that thought.
Speaker 1
I think this is an argument that often comes up as a counter argument to this idea that though this just won't be tracked by the relevant statistical agencies. And I think my first response is like, you know, why won't they want to track this? Like this surely seems like a very important thing that's happening in the world that governments should be interested in trying to track. And so if you have someone like an agency that is kind of really trying its best to track what's happening, then it should be pretty hard to be able to predict in advance the direction of the measurement error that they will obtain. maybe, you know, I think a good way of thinking about this is that there just might be noise rather than systematic bias. And insofar as that's the case, and you might indeed get lower growth rates, you might also get higher growth rates because it's just a noisier process. Now, there could be arguments that there's a systematic bias towards digital goods and, you know, Wikipedia and Google search not being tracked in maybe the right ways. My reading of that literature is that indeed there is some miss measurement. There are processes for updating and tracking kind of digital goods that happen. There are papers that look at revisions of these growth of these output estimates when people look at more carefully at these digital goods. And that suggests that there isn't this very large systematic bias or there isn't much of a systematic bias at all. And even if there was, I would say that these types of things are pretty small relative to the overall economy. And so maybe it's not surprising that there might be some issue with tracking it, might take time for people to adopt the process. There might not be very strong incentives to do so because it's a kind of small fraction of the economy.