The Data Exchange with Ben Lorica cover image

Exhaustion of High-Quality Data Could Slow Down AI Progress in Coming Decades

The Data Exchange with Ben Lorica

00:00

The Trends in Model Size

So model size trends. I mean, I guess we just keep getting bigger. Is that what you're is that the prediction generally of epoch? I guess, depends on the data type, right? And basically algorithmic improvement has been halving the compute requirements to reach a certain accuracy for, for example, for vision. So even if Moore's law stops completely just by getting more efficient algorithms, we might probably still scale. But that's only for vision. What about language? For language, it's a bit more blurry. We're also about to release a paper on that. It seems that returns to increasing size are currently lower than returns to increasing data. That means every time you

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app