Can chip efficiency slow AI's energy demand?

33 snips

Jul 18, 2024

Former Microsoft VP Christian Belady discusses the energy demand impact of AI data centers, exploring chip efficiency improvements, Moore’s Law limits, and constraints on AI growth. They examine the rise in data center energy consumption driven by AI boom, predicting 9% of U.S. power by 2030, and discuss potential solutions like changing computing architecture for energy savings.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Efficiency Can Increase Consumption

If a resource becomes cheaper, consumption often rises, known as Jevons Paradox.
Improving chip efficiency may increase total computing and thus energy use rather than decrease it.

INSIGHT

Data Center Energy Breakdown

About 50% of a typical data center's energy is consumed by CPUs and 30% by memory.
Networking and power conversion consume smaller portions but grow as complexity increases.

INSIGHT

GPUs and Moore's Law Limits

GPUs consume significantly more power than CPUs, now up to 1000 watts per module.
Moore's Law is slowing and chips are now multi-chip modules to maintain performance gains.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In March, Nvidia announced a new microchip designed for AI that is 25 times more energy efficient than its predecessor. Two months later, Google announced one with a 67% efficiency improvement. Today, the rest of the semiconductor industry is hyper focused on efficiency gains.

Will they save us from ballooning data center energy demands?

In this episode, Shayle talks to Christian Belady, former Microsoft vice president now focusing on data center advanced development. They unpack concerns about this new surge of demand and whether it’s different from the energy scare two decades ago. Back in 1999, researchers predicted that data centers could end up consuming half of U.S. electricity. But instead, demand remained largely flat at about 4% as cutting-edge hyperscale cloud computing displaced inefficient, on-premises servers.

And yet, driven by the AI boom, energy concerns are back. The Electric Power Research Institute predicts that data center loads could consume 9% of U.S. power generation by 2030. Demand is already rising fast, with emissions at both Google and Microsoft up significantly.

Shayle and Christian examine the factors driving those trends and what we can do about it, covering topics like:

Whether chip efficiency improvements will lead to energy savings or just more powerful computing
The upper limits of Moore’s Law
Energy, labor, and other big constraints on AI growth
Changing computing architecture to find energy savings
Enlisting data centers in integrated, or compulsory, demand response
Using AI to improve chip design

Recommended resources

Fierce Electronics: Power-hungry AI chips face a reckoning, as chipmakers promise ‘efficiency’

Latitude Media: The data center of the future looks like a massive virtual power plant

Latitude Media: Enchanted Rock is selling utilities on flexible data center connection

Latitude Media: Energy is now the ‘primary bottleneck’ for AI

Catalyst: Under the hood of data center power demand

Catalyst is brought to you by Kraken, the advanced operating system for energy. Kraken is helping utilities offer excellent customer service and develop innovative products and tariffs through the connection and optimization of smart home energy assets. Already licensed by major players across the globe, including Origin Energy, E.ON, and EDF, Kraken can help you create a smarter, greener grid. Visit kraken.tech.

Catalyst is brought to you by Anza Renewables, a data, technology, and services platform for solar and storage buyers. Anza’s real-time market intel equips buyers with the essential data they need to get the best deals. Download Anza’s free Q2 Module Pricing Insights Report at go.anzarenewables.com/latitude.

Catalyst is brought to you by Antenna Group, the global leader in integrated marketing, public relations, creative, and public affairs for energy and climate brands. If you're a startup, investor, or enterprise that's trying to make a name for yourself, Antenna Group's team of industry insiders is ready to help tell your story and accelerate your growth engine. Learn more at antennagroup.com.