Scaling AI for the Coming Data Deluge

Jul 19, 2024

Robert Nishihara, Anyscale CEO, discusses scaling AI models, generative AI's impact on enterprise interest, and the importance of quick deployment. The conversation covers challenges of handling massive amounts of data, evolving AI workloads, and the transition to multimodal AI models integrating text, audio, video, and images.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

Berkeley Lab Boosted Ray's Creation

Robert Nishihara credits the Berkeley Sky Computing Lab for their unique mix of AI and systems expertise.
This collaboration was crucial for building Ray, benefiting from lessons learned by creators of Spark and other systems.

INSIGHT

AI Workloads Are Data Intensive

AI workloads are becoming both GPU intensive and highly data intensive, especially with video and multimodal data.
Existing systems built just for compute-intensive tasks struggle to handle this new regime efficiently.

INSIGHT

Generative AI Demands Scale and Data

Generative AI has made distributed computing vital for AI workloads, ending the era when single machine computing sufficed.
Data preparation and high-quality training data acquisition now demand as much compute as model training itself.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this episode of the AI + a16z podcast, Anyscale cofounder and CEO Robert Nishihara joins a16z's Jennifer Li and Derrick Harris to discuss the challenges of training and running AI models at scale; how a focus on video models — and the huge amount of data involved — will change generative AI models and infrastructure; and the unique experience of launching a company out of the UC-Berkeley Sky Computing Lab (the successor to RISElab and AMPLab).

Here's a sample of the discussion, where Robert explains how generative AI has turbocharged the appetite for AI capabilities within enterprise customers:

"Two years ago, we would talk to companies, prospective customers, and AI just wasn't a priority. It certainly wasn't a company-level priority in the way that it is today. And generative AI is the reason a lot of companies now reach out to us . . . because they know that succeeding with AI is essential for their businesses, it's essential for their competitive advantage.

"And time to market matters for them. They don't want to spend a year hiring an AI infrastructure team, building up a 20-person team to build all of the internal infrastructure, just to be able to start to use generative AI. That's something they want to do today."

At another point in the discussion, he notes on this same topic:

"One dimension where we try to go really deep is on the developer experience and just enabling developers to be more productive. This is a complaint we hear all the time with machine learning teams or infrastructure teams: They'll say that they hired all these machine learning people, but then the machine learning people are spending all of their time managing clusters or working on the infrastructure. Or they'll say that it takes 6 weeks or 12 weeks to get a model to transition from development to production . . . Or moving from a laptop to the cloud, and to go from single machine to scaling — these are expensive handoffs often involve rewriting a bunch of code."

Learn more:

Anyscale

Sky Computing Lab

Ray

Follow everyone on X:

Robert Nishihara

Jennifer Li

Derrick Harris

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.