Eye On A.I. #239 Tuhin Srivatsa: How Baseten is Disrupting AI Deployment & Scaling in 2025
Feb 26, 2025
Tuhin Srivatsa is the CEO and Co-founder of Baseten, a company revolutionizing machine learning infrastructure. In this engaging discussion, he tackles the broken nature of AI deployment and how Baseten is streamlining the process for enterprises. Discover the shift towards open-source AI and why it's a game-changer. Tuhin also highlights the hidden costs of AI inference, the reasons many models fail in production, and the exciting future of scalable AI infrastructure. This insightful conversation is a must-listen for those navigating the AI landscape.
AI Snips
Chapters
Transcript
Episode notes
DeepSeek Use Case
- A company using OpenAI wants to scale and seeks more control, transparency, and cost-effectiveness.
- DeepSeek offers an open-source model, enabling faster, cheaper, and more controlled model operation within the company's VPC.
Deploying Models with Baseten
- Deploy large models easily with Baseten's Trust: write ~20 lines of Python, push, and get a scalable API endpoint.
- Baseten automates scaling and provides observability tools for efficient model management.
Baseten's Cost Efficiency
- Baseten offers cost savings through elastic compute, performance tuning (distillation, speculative decoding), and software layer pricing.
- They negotiate compute costs for customers and enable scaling with traffic, optimizing resource utilization.