No Priors: Artificial Intelligence | Technology | Startups

The Future is Small Models, with Matei Zaharia, CTO of Databricks

Apr 6, 2023

Matei Zaharia, Cofounder and Chief Technologist at Databricks, shares insights on the evolution of AI and small models like Dolly. With just $30 and one server, anyone can now create capable AI models. Zaharia discusses the shift away from massive data sets and manual training annotations. He reflects on his journey from a PhD student working on Spark to leading a critical data infrastructure company. The episode also explores the future of AI in tech decision-making and the importance of streamlined data management.

39:29

Creator website

Episode guests

Matei Zaharia

Highlights

AI Chapters

Episode notes

How Fine Tuning Influences the Accuracy of Neural Machine Translation Models

01:17

How to differentiate

03:23

Democratizing sentiment analysis with chat GPD

01:27

Intro

2min

The Evolution of Databricks and AI Language Models

23min

Navigating the Transition from Academia to Startup Foundership

3min

Navigating Software Deployment and AI Trends

2min

Exploring the Limitations of Scaling in Machine Learning

4min

Navigating AI's Future in Tech Decision-Making

5min

If you have 30 dollars, a few hours, and one server, then you are ready to create a ChatGPT-like model that can do what’s known as instruction-following. Databricks’ latest launch, Dolly, foreshadows a potential move in the industry toward smaller and more accessible but extremely capable AIs. Plus, Dolly is open source, requires less computing power, and fewer data parameters than its counterparts.

Matei Zaharia, Cofounder & Chief Technologist at Databricks, joins Sarah and Elad to talk about how big data sets actually need to be, why manual annotation is becoming less necessary to train some models, and how he went from a Berkeley PhD student with a little project called Spark to the founder of a company that is now critical data infrastructure that’s increasingly moving into AI.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

Show Notes:

[01:29] - Origin of Databricks

[4:30] - Work at Stanford Lab

[5:29] - Dolly and Role of Open Source

[12:30] - Industry focus on high parameter count, understanding reasoning at small model scale

[18:42] - Enterprise applications for Dolly & chat bots

[25:06] - Making bets as an academic turned CTO

[36:23] - The early stages of AI and future predictions

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.