Practical AI: Machine Learning, Data Science, LLM cover image

Practical AI: Machine Learning, Data Science, LLM

Self-hosting & scaling models

Oct 31, 2023
41:09
Snipd AI
Tuhin, specializes in model deployment and monitoring at any scale, joins the show to talk about self-hosting open access models. They discuss trends in tooling and usage of open access models, common use cases for integrating self-hosted models, and the influence of generative AI on the ecosystem.
Read more

Podcast summary created with Snipd AI

Quick takeaways

  • Base 10 offers a scalable infrastructure for deploying and hosting AI models, abstracting away complexities of scaling, observability, and security.
  • Open source models have revolutionized the AI landscape, enabling widespread adoption across industries, but managing them requires careful consideration of latency, security, privacy, and cost.

Deep dives

Base 10 focuses on infrastructure challenges of hosting AI models

Base 10 provides solutions to the infrastructure challenges of hosting AI models in production. They offer an open source library called Trust that simplifies the process of containerizing models, handling version management, and providing workflow support. Base 10 aims to save users time and effort by abstracting away the complexities of scaling, observability, and security. They also enable A/B testing, facilitate model rollbacks, and offer features such as logging and observability. Base 10's approach helps users achieve faster time-to-production and maintain production-grade inference.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode