Odd Lots How to Build the Ultimate GPU Cloud to Power AI
65 snips
Jul 20, 2023 
Brannin McBee, co-founder of CoreWeave, shares insights on building a GPU cloud for AI. He discusses the intense demand for AI chips like NVIDIA's H100 and the challenges of acquiring them in a competitive landscape. The conversation dives into the evolution of data centers, infrastructure needs for AI advancements, and strategic pivots from crypto mining to AI technologies. Brannin also addresses how smaller companies can succeed in the crowded AI cloud services market despite the dominance of larger players.
AI Snips
Chapters
Transcript
Episode notes
Data Center Tiers
- Data centers are tiered based on reliability and uptime.
- Higher tiers, like three and four, offer better redundancy in power, cooling, and internet connectivity.
Data Center Capacity Crunch
- GPU compute's higher power density is causing data center capacity issues.
- This creates challenges with cooling and power distribution within the data center.
Connecting GPUs
- Legacy data centers use Ethernet for basic server communication.
- Modern AI infrastructure uses InfiniBand for high-throughput data transfer between GPUs.