Super Data Science: ML & AI Podcast with Jon Krohn

881: Beyond GPUs: The Power of Custom AI Accelerators, with Emily Webber

Apr 22, 2025

Emily Webber, Principal Solutions Architect at AWS, shares insights into cutting-edge AI hardware. She discusses the impressive specs of Trainium and Trainium2, highlighting how they outperform traditional GPUs. Learn about the AWS Neuron SDK and its customization options for AI frameworks. Emily explains the Nitro System's role in enhancing cloud performance and security. Listeners will discover the criteria for choosing between AWS’s accelerators and the importance of tailored AI infrastructure for effective model training.

01:17:06

Episode guests

Emily Webber

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Emily Webber emphasizes that AWS Trainium2 significantly enhances AI model training, offering four times the compute power of its predecessor.

The AWS Neuron SDK simplifies advanced AI training integration, allowing developers to optimize models without being tied to hardware intricacies.

AWS is investing $110 million in the Build on Tranium program to foster academic research, aiming to elevate real-world AI applications.

Deep dives

Revolutionizing Cloud Computing

Annapurna Labs has transformed cloud computing through its innovative hardware solutions. The team, responsible for designing the Tranium and Inferentia chips, has focused on creating efficient tools for training and deploying AI models really well. With the Nitro system architecture, they decoupled controls for data and instances, enhancing both scalability and security. This foundational change has laid the groundwork for AWS's competitive edge in the rapidly evolving AI landscape.

Intro

3min

From Finance to AI: A Journey into Machine Learning

12min

Exploring AI Accelerators and the Concept of Kernels in Programming

3min

Harnessing Custom AI Accelerators

27min

Understanding Tensor Parallelism in AI Models

2min

Optimizing TPUs for AI Advancements

8min

Navigating Instance Types for Machine Learning on AWS

4min

Insights on AI and Literature

14min

Exploring the Convergence of Meditation, AI Hardware, and Cloud Innovations

4min

Emily Webber speaks to Jon Krohn about her work at Amazon Web Services, from its Annapurna Labs-developed Nitro System, a foundational technology that can enhance securities and performance in the cloud and how Trainium2 became AWS’ most powerful AI chip with four times the compute of Trainium. Hear the specs of AWS’s chips and when to use them.

Additional materials: www.superdatascience.com/881

This episode is brought to you by ODSC, the Open Data Science Conference.

Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.

In this episode you will learn:

(08:36) Emily’s work on AWS’ SageMaker and Trainium
(23:54) How AWS Neuron lets builders tailor their approach to using frameworks
(29:07) Why using an accelerator is better than using a GPU
(35:29) The key differences between AWS Trainium and AWS Trainium2
(52:45) How to select between AWS Trainium and AWS Trainium2

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Super Data Science: ML & AI Podcast with Jon Krohn

881: Beyond GPUs: The Power of Custom AI Accelerators, with Emily Webber

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

Revolutionizing Cloud Computing

Advancements in AI with Tranium 2

Empowering Academic Research

The Role of AWS Neuron SDK

Optimizing Model Training

Remember Everything You Learn from Podcasts