Advancements and Concerns in GPU Technology for AI

This last week has been a big week for AI news. BigScience is training a huge language model (while the world watches), and NVIDIA announced their latest “Hopper” GPUs. Chris and Daniel discuss these and other topics on this fully connected episode!

Join the discussion

Changelog++ members get a bonus 2 minutes at the end of this episode and zero ads. Join today!

Sponsors:

• Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
• Changelog++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this! 

Featuring:

• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

BigScience Language Model Training

• Model Details
• Live model logs and training curves

H100 from NVIDIA

• [White paper](https://www.nvidia.com/en-us/data-center/h100/
   https://cf-store.widencdn.net/nvdam/8/4/a/84ae26c1-6774-4083-a41d-4f3d82a6a449.pdf)
• Omniverse platform
• Earth-2 Supercomputer

Stanford’s AI Index 2022

Something missing or broken? PRs welcome!