Practical AI Self-hosting & scaling models
10 snips
Oct 31, 2023 Tuhin Srivastava, Co-founder and CEO of Baseten, dives into the world of self-hosting open access models. He discusses common use cases for deploying these models while addressing the shift towards self-hosting solutions. The conversation explores the evolution of machine learning infrastructure, the challenges of model deployment, and the importance of data privacy. Tuhin also highlights innovative advancements in model hosting frameworks and the growing significance of multi-cloud strategies in AI deployment.
AI Snips
Chapters
Transcript
Episode notes
Democratization of AI
- Open-source models and the rise of platforms like Hugging Face have democratized AI.
- This shift has empowered engineers to utilize ML, expanding the user base beyond data scientists.
High Expectations in AI
- The high expectations set by ChatGPT have raised the bar for AI product development.
- Users now demand faster, more sophisticated models, making production-level inference crucial.
Drone Crash Anecdote
- Tuhin Srivastava shared an anecdote about crashing a drone after switching from autopilot to manual mode.
- This story highlighted the challenges of managing complex systems without proper understanding.