Navigating the Challenges of Running Open Source Machine Learning Models

Today we have Hassan back on the show. Hassan was one of our first guests for Huddle when he was working at Vercel, but since then, he's joined Together AI, one of the hottest companies in the world. They just raised a massive series B round.

Hassan joins us to talk about Together AI, inference optimization and building AI applications. We touch on a bunch of topics like customer uses of AI, best practices for building apps, and what's next for Together AI.

Timestamps

01:42 Opportunity at Together AI

04:26 Together raised a big round

06:06 Vision Behind Together AI

08:32 Problems in running Open Source Models

11:40 Speed For Inference

14:24 Fine Tuning

19:23 One or Two Models or a Combination of them

21:32 Serverless

22:21 Cold Start issues?

27:46 How much data do you need?

30:00 Balancing Reliability and Cost

34:07 How customers are using Together

42:36 Agent Recipes

47:03 Typical Mistakes buiilding AI apps