Power Law with John Coogan Interview with Gavin Uberti & Robert Wachen (Etched)
56 snips
Jun 25, 2024 AI experts Gavin Uberti & Robert Wachen discuss advancements in AI models, custom chip design impact on job market, and scaling challenges in building data centers. They explore the future of GPU technology, government regulations, and the evolution of hardware/software in AI model training.
AI Snips
Chapters
Transcript
Episode notes
Specialized Chips Beat General GPUs
- Etched builds specialized chips for Transformers to gain an order-of-magnitude efficiency over general-purpose GPUs.
- When models are enormous, bespoke silicon and high utilization materially improve economics and throughput.
The Bitter Lesson Meets Memory Bottlenecks
- The "bitter lesson" shows scale and compute drive progress, but most GPU silicon area moves memory not flops.
- Custom chips can reallocate area toward the compute patterns Transformers actually need.
Secure Emulation Capacity Early
- Buy or secure FPGA/emulator access early because evaluating big AI chips on FPGAs is costly and capacity-limited.
- Negotiate supplier pricing aggressively to reduce emulator and prototyping spend.