Ion Stoica, a professor at UC Berkeley and co-founder of game-changers like Conviva and Databricks, shares insights on transforming research into successful companies. He reflects on the inception of Apache Spark and its pivotal role in Databricks' rise. Stoica discusses the unique challenges faced in building Anyscale and why AI’s integration across industries is crucial. He also emphasizes the importance of market positioning and evolving tech landscapes, predicting significant trends in AI's future adoption.
01:03:11
forum Ask episode
web_stories AI Snips
view_agenda Chapters
auto_awesome Transcript
info_circle Episode notes
volunteer_activism ADVICE
Win Ecosystem Before Monetizing
Prioritize ecosystem adoption before monetization for new technologies.
Aim to make the best product to attract users organically.
question_answer ANECDOTE
Choosing Academia for Flexibility
Ion chose academia first for flexibility and optionality despite the tech boom.
Returning to startups later was easier after establishing academic foundations.
question_answer ANECDOTE
Spark's Origin in Memory Speed
Spark arose to solve slow, disk-heavy machine learning tasks by keeping data in memory.
This in-memory design drastically sped up iterative ML workloads compared to Hadoop.
Get the Snipd Podcast app to discover more snips from this episode
My guest today is Ion Stoica, professor of computer science at UC Berkeley and the co-founder of Conviva, Databricks, and Anyscale. Over the last two decades, Ion’s research labs - the AMP Lab, the RISE Lab, and now the Sky Computing Lab - have seeded a generation of category-defining companies.
Ion has the unique ability to turn non-consensus ideas into durable businesses. He applied machine learning to video optimization with Conviva before AI became mainstream. He scaled Apache Spark into a $60B platform with Databricks. And now, with Anyscale, he’s betting on Ray as the foundation for distributed AI workloads.
In this episode, we dig into both sides of Ion’s work: how to build world-class research labs, and how to turn research into real companies. His clarity of thought makes the future feel legible, and his track record suggests he’s very often right.
Hope you enjoy the conversation!
Chapters:
00:00 The Spark thesis: win the ecosystem first, monetize later
01:00 Intro: From lab to company - Ion’s repeatable playbook
03:00 Did you always plan to become a founder, or did it just happen?
05:23 Let’s start with Spark - how did the project come about?
13:04 What were the most important early decisions at Databricks?
23:49 You were the first CEO - what did you have to learn (or unlearn)?
30:01 How was building Anyscale different from building Databricks?
33:53 What’s obvious to you about the future of AI that others miss?
B2BaCEO (with Ashu Garg)