No Priors: Artificial Intelligence | Technology | Startups Speed will win the AI computing battle with Tuhin Srivastava from Baseten
40 snips
Mar 21, 2024 Tuhin Srivastava, CEO and co-founder of Baseten, specializes in scalable AI infrastructure. He shares insights on why speed is crucial for AI development, emphasizing the need for efficient code rather than no-code solutions. The conversation highlights surprising use cases for Baseten and discusses the challenges of AI training and deployment in the current landscape. Tuhin also tackles the impact of hardware shortages and the defensibility of jobs in AI, arguing that strategic investment in computing is essential for future success.
AI Snips
Chapters
Transcript
Episode notes
Training vs. Inference Workloads
- Training workloads prioritize co-locating GPUs and network optimization.
- Inference workloads prioritize co-location with user activity and repeatable workflows, favoring resiliency and reliability.
Baseten's Surprising Growth
- Baseten saw unexpected market acceleration in late 2022 and 2023 after a quiet period.
- Teams prioritize speed in AI, leading to increased buy vs. build decisions for infrastructure.
Optimizing Inference Performance
- Inference optimization involves maximizing GPU usage, scaling across GPUs, and staying up-to-date with open-source advancements.
- Baseten's partnership with NVIDIA and TRT-LLM has driven performance gains, focusing on low-level optimizations and open-source contributions.