The Lawfare Podcast Scaling Laws: AI and Energy: What Do We Know? What Are We Learning?
Oct 17, 2025
Mosharaf Chowdhury, an energy optimization expert from the University of Michigan, and AI researcher Dan Zhou share insights on the energy demands of AI systems. They delve into the substantial energy costs of a single ChatGPT query and examine the surprising dominance of inference energy over training. The duo discusses how GPU inefficiencies and rising demands complicate energy use, while also exploring methods to enhance efficiency and the need for clearer public communication on AI's environmental impact.
AI Snips
Chapters
Transcript
Episode notes
Inference, Not Training, Drives Long-Run Energy
- Training happens infrequently while inference scales with millions of users and dominates long-run energy use.
- Inference energy per query is small but aggregates to more than training as usage grows massively.
GPU Inefficiency Is A Major Waste Source
- GPUs consume power even idle and require careful batching and overlap to be efficient.
- Novice users often run GPUs inefficiently, increasing aggregate energy waste.
MIT Power Cap Showed Rebound Effect
- MIT implemented a GPU power cap that saved energy without perceptible slowdown for users.
- Users then risked running more jobs because saved capacity lowered marginal cost and demand rebounded.