This Week in Startups

CoreWeave’s Brannin McBee on the future of AI infrastructure, GPU economics, & data centers | E1925

Apr 4, 2024

Brannin McBee, Co-founder and CDO of CoreWeave, shares insights from transitioning his company from cryptocurrency to AI infrastructure. He discusses the soaring energy demands of GPUs, emphasizing innovative cooling solutions and the significant costs tied to GPU operations. The conversation highlights CoreWeave's expansion with new data centers to tackle surging AI demands, and the competition in AI hardware, particularly against NVIDIA's dominance. McBee also explores the evolving landscape of AI applications and their implications for industries like advertising and e-commerce.

55:56

Creator website

Episode guests

Brannin McBee

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

AI software adoption outpaces infrastructure capabilities, creating challenges in meeting demand.

GPU clusters are crucial for high-performance computing in AI workloads, driving the need for massive infrastructure capacity.

AI revolution reshapes sectors like advertising, emphasizing the demand for scalable infrastructure to power personalized ad strategies.

Deep dives

Challenges of Current Cloud Infrastructure

The existing cloud infrastructure developed over the last decade wasn't designed for parallelizable workloads, creating a need to rebuild physical infrastructure at the pace of AI software adoption. Brandon McBee from CoreWeave highlighted that AI software adoption is outpacing infrastructure capabilities, leading to challenges in keeping up with demand.

Snip

NaN:NaN

Intro

2min

From Crypto to AI: A Strategic Shift

10min

Navigating the Economics of Large-Scale GPU Infrastructure

4min

Powering AI: The GPU Energy Dilemma

13min

The Evolution of AI Hardware: GPUs vs. LPUs

3min

NVIDIA's Reign in AI Infrastructure

5min

Navigating Data Throughput Challenges in AI Infrastructure

4min

Navigating AI Infrastructure and Advertising Evolution

14min

This Week in Startups is brought to you by…

OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. TWiST listeners can get an extra 20% off any plan for your first 6 months at http://www.openphone.com/twist

Gusto is easy online payroll, benefits, and HR built for modern small businesses. Get three months free when you run your first payroll at Gusto.com/twist.

*Gusto pricing shown in ad is based on pricing prior to March 2025

Northwest Registered Agent. Northwest Registered Agent will form your business quickly and easily. For just $39 plus state fees, Northwest will handle your complete business identity. Visit https://www.northwestregisteredagent.com/twist⁠ today.

Todays show:

Brannin McBee joins Jason to discuss CoreWeave’s transition from cryptocurrency to AI (3:36), the energy dynamics of GPUs (14:04), innovations in data center cooling (19:28), and the future challenges and impacts of AI infrastructure on industries like search, advertising, and e-commerce (45:36).

Timestamps:

(00:00) CoreWeave’s Brannin McBee joins Jason

(3:36) The founding story of CoreWeave and the transition from the cryptocurrency market to AI

(12:49) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist

(14:04) Energy reliance of GPUs, the future of data centers, and the cost and efficiency of GPU cloud infrastructure

(19:28) Direct-to-chip liquid cooling and other cooling solutions for GPUs

(24:52) Demand trends for GPU capacity and the growth of inference linked to user growth

(29:06) Gusto - Get three months free when you run your first payroll at http://gusto.com/twist

(30:28) LPUs Vs. GPUs

(34:13) Dominance of Nvidia in training models, implications of open source chips and chip architecture

(39:01) Northwest Registered Agent - For just $39 plus state fees, Northwest will handle your complete business identity. Visit ⁠https://www.northwestregisteredagent.com/twist⁠ today.

(40:00) Challenges in training large language models and the use of infrastructure as a weapon by big tech

(45:36) The potential impact of AI on search engines, advertising sector, and ecommerce

(52:46) Timeline for supply-demand balance in AI infrastructure and the operational feat of running multiple data centers

Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp

Follow Brannin:

X: https://twitter.com/branninmcbee

LinkedIn: https://www.linkedin.com/in/branninmcbee

Follow Jason:

X: https://twitter.com/Jason

LinkedIn: https://www.linkedin.com/in/jasoncalacanis

Thank you to our partners:

(12:49) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist

(29:06) Gusto - Get three months free when you run your first payroll at http://gusto.com/twist

*Gusto pricing shown in ad is based on pricing prior to March 2025

(39:01) Northwest Registered Agent - For just $39 plus state fees, Northwest will handle your complete business identity. Visit ⁠⁠https://www.northwestregisteredagent.com/twist⁠⁠ today.|

Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland

Check out Jason’s suite of newsletters: https://substack.com/@calacanis

Follow TWiST:

Substack: https://twistartups.substack.com

Twitter: https://twitter.com/TWiStartups

YouTube: https://www.youtube.com/thisweekin

Instagram: https://www.instagram.com/thisweekinstartups

TikTok: https://www.tiktok.com/@thisweekinstartups

Subscribe to the Founder University Podcast: https://www.founder.university/podcast