The Nonlinear Library EA - AI scaling myths by Nicholas Kruus
Jun 29, 2024
01:02
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI scaling myths, published by Nicholas Kruus on June 29, 2024 on The Effective Altruism Forum.
"So far, bigger and bigger language models have proven more and more capable. But does the past predict the future?
"One popular view is that we should expect the trends that have held so far to continue for many more orders of magnitude, and that it will potentially get us to artificial general intelligence, or AGI.
"This view rests on a series of myths and misconceptions. The seeming predictability of scaling is a misunderstanding of what research has shown. Besides, there are signs that LLM developers are already at the limit of high-quality training data. And the industry is seeing strong downward pressure on model size. While we can't predict exactly how far AI will advance through scaling, we think there's virtually no chance that scaling alone will lead to AGI..."
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
"So far, bigger and bigger language models have proven more and more capable. But does the past predict the future?
"One popular view is that we should expect the trends that have held so far to continue for many more orders of magnitude, and that it will potentially get us to artificial general intelligence, or AGI.
"This view rests on a series of myths and misconceptions. The seeming predictability of scaling is a misunderstanding of what research has shown. Besides, there are signs that LLM developers are already at the limit of high-quality training data. And the industry is seeing strong downward pressure on model size. While we can't predict exactly how far AI will advance through scaling, we think there's virtually no chance that scaling alone will lead to AGI..."
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org