Software Snack Bites

Nvidia, Groq, HBM & What's Ahead for Semis (Doug O'Laughlin, Founder of Fabricated Knowledge)

Apr 9, 2024

Doug O'Laughlin, Founder of Fabricated Knowledge, discusses semiconductors in AI, emphasizing the importance of understanding the semiconductor level for projecting AI trends. Topics include inference chips (Groq & Cerebras), HBM's importance, Nvidia's strategy, ASICs, Broadcom, and other semis trends. Doug's insights have led to significant gains in Nvidia and Intel. The podcast also explores model stabilization, GPU evolution, networking technologies, investment strategies, and anticipation for technological events.

01:03:41

Creator website

Episode guests

Doug O'Laughlin

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Specialization of chips for AI is crucial for future advancements.

NVIDIA's strategic move with CUDA has influenced AI technology development.

Transition towards utilizing HBM for higher bandwidth memory is significant in semiconductor industry.

Deep dives

Specialization of Chips for AI: GPUs vs. ASICS

The podcast delves into the specialization of chips for AI, particularly GPUs and ASICS. It discusses how ASICS may gain more prominence if model innovation stagnates. The comparison between the rigidity of ASICS and the flexibility of GPUs is highlighted as crucial, especially during periods of rapid change.

Introduction

2min

Evolution of GPUs and Semiconductor Chip Design

18min

Evolution of Networking Technologies in GPU Clusters

28min

Emerging Trends in Semiconductors: HBM and Hybrid Bonding

3min

Intel's Breakthrough Backside Power Delivery Method

5min

Investment Strategies and Industry Cycles

6min

Anticipation for Technological Events and Research Support

2min

The most important trend powering AI is semiconductors and that trend is not slowing down anytime soon. We need to understand what is happening at the semiconductor level to be able to project forward what we think will occur in AI. This episode is the best way to learn what’s ahead of semis.

Doug O’Laughlin is the first 2nd time guest on Software Snack Bites! He is the Founder of Fabricated Knowledge which is one of the fastest growing and highest grossing newsletters out there. The reason for that is because each of his write-ups is packed with insights that even a layperson can understand. We did this podcast right before Nvidia GTC so you’ll have to read Fabricated Knowledge to get Doug’s takes on the new announcements. In this podcast we talk about the inference specific chips (Groq & Cerebras), why HBM (high bandwidth memory) is so important to the future semi story, Nvidia’s current dominant strategy and why they are well positioned to continue winning, where ASICs and Broadcom fit in, and some other trends outside of semis Doug is watching.

If you had listened to Doug’s episode last year, you would be up 250% on Nvidia. If you had read his writing, you would be up even more. Additionally, if you read his writing on Intel, you would have had a 65% gain in < 6 months.

Of course, this is not investment advice, do your own research, but Doug’s knowledge will at least help you figure out where to start your diligence. There are multiple links below with terminology discussed and Doug’s free articles on certain topics. The best way to support him is to subscribe to Fabricated Knowledge.

Where to Find Doug:

* LinkedIn: https://www.linkedin.com/in/dougolaughlin/

* Twitter: https://twitter.com/FoolAllTheTime

* Semiconductor Publication: https://www.fabricatedknowledge.com

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(02:06) – Why GPUs & CUDA are So Important to LLMs

(03:58) – Why GPUs Are More Specialized Than We Think

(06:47) – The Incentive for AI Teams to Stay with Nvidia