AI Snips
Chapters
Transcript
Episode notes
Nuro Experience Shapes Codeium
- Varun and Anshul both worked at autonomous vehicle company Nuro, where they faced large scale ML workload challenges.
- Their shared background in GPU and ML infrastructure led them to found Exa Function and later build Codeium focusing on generative AI.
GPU Virtualization Challenges
- GPUs are scarce and expensive, and unlike CPUs, cannot be easily virtualized to run many workloads.
- Efficient GPU usage is crucial since modern high-end GPUs cost tens of thousands of dollars each.
Explosive Growth of AI GPU Workloads
- AI workloads on GPUs have grown exponentially in size and scale since 2018.
- Managing over 10,000 GPUs in a single cloud region shows the rapid growth and importance of deep learning infrastructure.