AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Trends in Model Size
So model size trends. I mean, I guess we just keep getting bigger. Is that what you're is that the prediction generally of epoch? I guess, depends on the data type, right? And basically algorithmic improvement has been halving the compute requirements to reach a certain accuracy for, for example, for vision. So even if Moore's law stops completely just by getting more efficient algorithms, we might probably still scale. But that's only for vision. What about language? For language, it's a bit more blurry. We're also about to release a paper on that. It seems that returns to increasing size are currently lower than returns to increasing data. That means every time you