The Impact of Social Information on Democracy

This week we talk about neural networks, AGI, and scaling laws.

We also discuss training data, user acquisition, and energy consumption.

Recommended Book: Through the Grapevine by Taylor N. Carlson

Transcript

Depending on whose numbers you use, and which industries and types of investment those numbers include, the global AI industry—that is, the industry focused on producing and selling artificial intelligence-based tools—is valued at something like a fifth to a quarter of a trillion dollars, as of halfway through 2024, and is expected to grow to several times that over the next handful of years, that estimate ranging from two or three times, to upward of ten or twenty-times the current value—again, depending on what numbers you track and how you extrapolate outward from those numbers.

That existing valuation, and that projected (or in some cases, hoped-for growth) is predicated in part on the explosive success of this industry, already.

It went from around $10 billion in global annual revenue in 2018 to nearly $100 billion in global revenue in 2024, and the big players in this space—among them OpenAI, which kicked off the most recent AI-related race, the one focusing on large-language models, or LLMs, when it released its ChatGPT tool at the tail-end of 2022—have been attracting customers at a remarkable rate, OpenAI hitting a million users in just five days, and pulling in more than 100 million monthly users by early 2023; a rate of customer acquisition that broke all sorts of records.

This industry’s compound annual growth rate is approaching 40%, and is expected to maintain a rate of something like 37% through 2030, which basically means it has a highly desirable rate of return on investment, especially compared to other potential investment targets.

And the market itself, separate from the income derived from that market, is expected to grow astonishingly fast due to the wide variety of applications that’re being found for AI tools; that market expanded by something like 50% year over year for the past five years, and is anticipated to continue growing by about 25% for at least the next several years, as more entities incorporate these tools into their setups, and as more, and more powerful tools are developed.

All of which paints a pretty flowery picture for AI-based tools, which justifies, in the minds of some analysts, at least, the high valuations many AI companies are receiving: just like many other types of tech companies, like social networks, crypto startups, and until recently at least, metaverse-oriented entities, AI companies are valued primarily based on their future potential outcomes, not what they’re doing today.

So while many such companies are already showing impressive numbers, their numbers five and ten years from now could be even higher, perhaps ridiculously so, if some predictions about their utility and use come to fruition, and that’s a big part of why their valuations are so astronomical compared to their current performance metrics.

The idea, then, is that basically every company on the planet, not to mention governments and militaries and other agencies and organizations will be able to amp-up their offerings, and deploy entirely new ones, saving all kinds of money while producing more of whatever it is they produce, by using these AI tools. And that could mean this becomes the industry to replace all other industries, or bare-minimum upon which all other industries become reliant; a bit like power companies, or increasingly, those that build and operate data centers.

There’s a burgeoning counter-narrative to this narrative, though, that suggests we might soon run into a wall with all of this, and that, consequently, some of these expectations, and thus, these future-facing valuations, might not be as solid as many players in this space hope or expect.

And that’s what I’d like to talk about today: AI scaling walls—what they are, and what they might mean for this industry, and all those other industries and entities that it touches.

—

In the world of artificial intelligence, artificial general intelligence, or AGI, is considered by many to be the ultimate end-goal of all the investment and application in and of these systems that we’re doing today.

The specifics of what AGI means varies based on who you talk to, but the idea is that an artificial general intelligence would be “generally” smart and capable in the same, or in a similar way, to human beings: not just great at doing math and not just great at folding proteins, or folding clothes, but pretty solid at most things, and trainable to be decent, or better than decent at potentially everything.

If you could develop such a model, that would allow you, in theory, to push humans out of the loop for just about every job: an AI bot could work the cash register at the grocery store, could drive all the taxis, and could do all of our astronomy research, to name just a few of the great many jobs these systems could take on, subbing in for human beings who would almost always be more expensive, but who—this AI being a generalist and pretty good at everything—wouldn’t necessarily do any better than these snazzy new AI systems.

So AGI is a big deal because of what it would represent in terms of us suddenly having a potentially equivalent intelligence, an equivalent non-human intelligence, to deal with and theorize over, but it would also be a big deal because it could more or less put everyone out of work, which would no doubt be immensely disruptive, but it would also be really, really great for the pocketbooks of all the companies that are currently burdened with all those paychecks they have to sign each month.

The general theory of neural network-based AI systems, which basically means software that is based in some way on the neural networks that biological entities, like mice and fruit flies and humans have in our brains and throughout our bodies, is that these networks should continue to scale as the number of factors that go into making them scale: and usually those factors include the size of the model—which in the case of most of these systems means the number of parameters it includes—the size of the dataset it trains on—which is the amount of data, written, visual, audio, and otherwise, that it’s fed as it’s being trained—and the amount of time and resources invested in its training—which is a variable sort of thing, as there are faster and slower methods for training, and there are more efficient ways to train that use less energy—but in general, more time and more resources will equal a more girthy, capable AI system.

So scale those things up and you’ll tend to get a bigger, up-scaled AI on the other side, which will tend to be more capable in a variety of ways; this is similar, in a way, to biological neural networks gaining more neurons, more connections between those neurons, and more life experience training those neurons and connections to help us understand the world, and be more capable of operating within it.

That’s been the theory for a long while, but the results from recent training sessions seem to be pouring cold water on that assumption, at least a bit, and at least in some circles.

One existing scaling concern in this space is that we, as a civilization, will simply run out of novel data to train these things on within a couple of years.

The pace at which modern models are being trained is extraordinary, and this is a big part of why the larger players, here, don’t even seriously talk about compensating the people and entities that created the writings and TV shows and music they scrape from the web and other archives of such things to train their systems: they are using basically all of it, and even the smallest payout would represent a significant portion of their total resources and future revenues; this might not be fair or even legal, then, but that’s a necessary sacrifice to build these models, according to the logic of this industry at the moment.

The concern that is emerging, here, is that because they’ve already basically scooped up all of the stuff we’ve ever made as a species, we’re on the verge of running out of new stuff, and that means future models won’t have more music and writing and whatnot to use—it’ll have to rely on more of the same, or, and this could be even worse, it’ll have to rely on the increasing volume of AI-generated content for future iterations, which could result in what’s sometimes called a “Habsburg AI,” referring to the consequences of inbreeding over the course of generations: and future models using AI-generated content as their source materials may produce distorted end-products that are less and less useful (and even intelligible) to humans, which in turn will make them less useful overall, despite technically being more powerful.

Another concern is related to the issue of physical infrastructure.

In short, global data centers, which run the internet, but also AI systems, are already using something like 1.5-2% of all the energy produced, globally, and AI, which use an estimated 33% more power to generate a paragraph of writing or an image, than task-specific software would consume to do the same, is expected to double that figure by 2025, due in part to the energetic costs of training new models, and in part to the cost of delivering results, like those produced by the ChatGPTs of the world, and those increasingly generated in lieu of traditional search results, like by Google’s AI offerings that’re often plastered at the top of their search results pages, these days.

There’s a chance that AI could also be used to reduce overall energy consumption in a variety of ways, and to increase the efficiency of energy grids and energy production facilities, by figuring out the optimal locations for solar panels and coming up with new materials that will increase the efficiency of energy transmission. But those are currently speculative benefits, and the current impact of AI on the energy grid is depletionary, not additive.

There’s a chance, then that we’ll simply run out of energy, especially on a local basis, where the training hubs are built, to train the newest and greatest and biggest models in the coming years. But we could also run out of other necessary resources, like the ginormous data centers required to do said training, and even the specific chips that are optimized for this purpose that are in increasingly short supply because of how vital this task has become for so many tech companies, globally.

The newest concern in this space, related to future growth, though, is related to what are called scaling laws, which refer to a variety of theories—some tested, some not yet fully tested—about how much growth you can expect if you use the same general AI system structure, and just keep pumping it up with more resources, training data, and training time.

The current batch of most powerful and, for many use-cases, most useful AI systems are the result of scaling basically the same AI system structure so that it becomes more powerful and capable over time. There’s delay between new generations because tweaks are made, all that training and feeding has to be done, but also because there are adjustments required afterward to optimize the system for different purposes and for stability.

But a slew of industry experts have been raising the alarm about a possible bubble in this space, not because it’s impossible to build more powerful AI, but because the majority of resources that have been pumped into the AI industry in recent years are basically just inflating a giant balloon predicated on scaling the same things over and over again, every company doing this scaling hoping to reach AGI or something close to AGI before their competitors, in order to justify those investments and their sprawling valuations.

In other words, it’s a race to a destination that they might not be able to reach, in the near-future, or ever, using the current batch of technologies and commonly exploited approaches, but they can’t afford to dabble in too many alternatives, at least not thoroughly, because there’s a chance if they take their eyes off the race they’re running, right now, one of their many also-super-well-funded opponents will get there first, and they’ll be able to make history, while also claiming the lion’s share of the profits, which could be as substantial as the entire economy, if you think of those aforementioned benefits of being able to replace a huge chunk of the world’s total employee base with equally capable bots.

The most common version of this argument, that the current generation of AI systems are hitting a point of diminishing returns—still growing and becoming more powerful as they scale, but not as much as anticipated, less growth and power per unit of resource, training time, size of dataset, and so on, compared to previous generations—and that diminishment means, according to this argument, we’ll continue to see a lot of impressive improvements, but should not longer expect the doubling of capability every 5 to 14 months that we’ve seen these past few years.

We’ve picked the low-hanging fruit, in other words, and everything from this point forward will be more expensive, less certain, and thus, less appealing to investors—while also potentially being less profitable, and thus, the money that’s been plowed into these businesses, thus far, might not payout, and we could see some large-scale collapses due to the disappearance of those resources that are currently funding this wave of AI-scaling, as a consequence.

If true, this would be very bad in a lot of ways, in part because these are resources that could have been invested in other things, and in part because a lot of hardware and know-how and governmental heft have been biased toward these systems for years now; so the black hole left behind, should all of that disappear or prove to be less than many people assumed, would be substantial, and could lead to larger-scale economic issues; that gaping void, that gravity well made worse because of those aforementioned sky-high valuations, which are predicated mostly on what these companies are expected to do in the future, not what they’re doing, today—so that would represent a lot of waste, and a lot of unrealized, but maybe never feasible in the first place, potential.

This space is maybe propped up by hype and outlandish expectations, in other words, and the most recent results from OpenAI and their upcoming model seem to lend this argument at least some credibility: the publicly divulged numbers only show a relatively moderate improvement over their previous core model, GPT4, and it’s been suggested, including by folks who previously ran OpenAI, that more optimizing after the fact, post-training, will be necessary to get the improvements the market and customers are expecting—which comes with its own unknowns and additional costs, alongside a lack of seemingly reliable, predictable scaling laws.

For their part, the folks currently at the top of the major AI companies have either ignored this line of theorizing, or said there are no walls, nothing to see here, folks, everything is going fine. 

Which could be true, but they’re also heavily motivated not to panic the market, so there’s no way to really know at this point how legit their counter-claims might be; there could be new developments we’re not currently, publicly aware of, but it could also be that they’re already working those post-training augmentations into their model of scaling, and just not mentioning that for financial reasons.

AI remains a truly remarkable component of the tech world, right now, in part because of what these systems have already shown themselves capable of, but also because of those potential, mostly theorized, at this point, benefits they could enable, across the economy, across the energy grid, and so on.

The near-future outcomes, though, will be interesting to watch, as it could be we’ll see a lot of fluffed-up models that roughly align with anticipated scaling-laws, but which didn’t get there by the expected, training-focused paths, which would continue to draw questions from investors who had specific ideas about how much it would cost to get what sorts of outcomes, which in turn would curse this segment of the economy and technological development with more precarious footing than it currently enjoys.

We might also see a renewed focus on how these systems are made available to users: a rethinking of the interfaces used, and the use-cases they’re optimized for, which could make the existing (and near-future) models ever more useful, despite not becoming as powerful as anticipated, and despite probably not getting meaningfully closer to AGI, in the process.

Show Notes

https://arxiv.org/abs/2311.16863

https://www.weforum.org/stories/2024/07/generative-ai-energy-emissions/

https://epochai.org/blog/will-we-run-out-of-ml-data-evidence-from-projecting-dataset

https://www.semafor.com/article/11/13/2024/tiktoks-new-trademark-filings-suggest-its-doubling-down-on-its-us-business

https://arxiv.org/abs/2001.08361

https://archive.ph/d24pA

https://www.fastcompany.com/91228329/a-funny-thing-happened-on-the-way-to-agi-model-supersizing-has-hit-a-wall

https://futurism.com/the-byte/openai-research-best-models-wrong-answers

https://en.wikipedia.org/wiki/Neural_network_(machine_learning)

https://en.wikipedia.org/wiki/Neural_scaling_law

https://futurism.com/the-byte/openai-research-best-models-wrong-answers

https://futurism.com/the-byte/ai-expert-crash-imminent

https://www.theverge.com/2024/10/25/24279600/google-next-gemini-ai-model-openai-december

https://ourworldindata.org/artificial-intelligence?insight=ai-hardware-production-especially-cpus-and-gpus-is-concentrated-in-a-few-key-countries

https://blogs.idc.com/2024/08/21/idcs-worldwide-ai-and-generative-ai-spending-industry-outlook/

https://explodingtopics.com/blog/chatgpt-users

https://explodingtopics.com/blog/ai-statistics

https://www.aiprm.com/ai-statistics/

https://www.forbes.com/advisor/business/ai-statistics/

https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai

https://www.researchgate.net/profile/Gissel-Velarde-2/publication/358028059_Artificial_Intelligence_Trends_and_Future_Scenarios_Relations_Between_Statistics_and_Opinions/links/61ec01748d338833e3895f80/Artificial-Intelligence-Trends-and-Future-Scenarios-Relations-Between-Statistics-and-Opinions.pdf

https://www.statista.com/outlook/tmo/artificial-intelligence/worldwide

https://en.wikipedia.org/wiki/Artificial_intelligence#Applications

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit letsknowthings.substack.com/subscribe