No Priors: Artificial Intelligence | Technology | Startups The marketplace for AI compute with Jared Quincy Davis from Foundry
13 snips
Aug 22, 2024 
Jared Quincy Davis, Founder and CEO of Foundry and former DeepMind researcher, dives into the fascinating world of AI cloud computing. He discusses the unique challenges of GPU utilization for large models and how Foundry is leading the charge in improving cloud economics. Jared also shares insights on the evolving GPU market and the complexities of designing compound AI systems. His predictions for the future of GPU capacity offer a glimpse into what innovations lie ahead in the AI landscape.
AI Snips
Chapters
Transcript
Episode notes
Small Teams, Big Impact
- AlphaFold 2 and ChatGPT, impactful AI projects, were developed by relatively small teams.
- These teams leveraged substantial resources like Google's infrastructure and OpenAI's compute budget.
GPU Utilization and Failures
- Even in dedicated GPU clusters, utilization is often below 80% due to failures and downtimes.
- Modern GPUs are complex systems with many components, increasing the likelihood of failures.
AI Cloud vs. True Cloud
- Current AI cloud offerings resemble co-location services, not true cloud computing.
- True cloud computing offers elasticity, allowing workloads to be reshaped and scaled dynamically.