The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Powering AI with the World's Largest Computer Chip with Joel Hestness - #684
May 13, 2024
In this discussion, Joel Hestness, a principal research scientist and lead of the core machine learning team at Cerebras, dives into the groundbreaking Wafer Scale Engine 3. He explains how this custom silicon surpasses traditional AI hardware, focusing on its unique architecture and memory capabilities. Joel also covers advancements in large language model training, innovative optimization techniques, and the integration of open-source frameworks like PyTorch. Additionally, he shares exciting research on weight-sparse training and novel optimizers that leverage higher-order statistics.
55:06
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Cerebras' Wafer Scale Engine 3 offers a unique AI hardware solution for large language models.
- Cerebras' software stack enables ultra-low latency for inference and supports deployment on various platforms.
Deep dives
Joel's Background in Machine Learning and Heterogeneous Processor Design
Joel Hessness discusses his journey from specializing in heterogeneous processor design during his PhD to working on large-scale language and speech recognition models at Baidu. His experience highlighted the need for scalable compute power for complex applications, particularly in machine learning.
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
