AI + a16z

Scoping the Enterprise LLM Market

Apr 12, 2024

Naveen Rao, VP of generative AI at Databricks, discusses enterprise LLMs and generative AI, highlighting the evolution of language models, challenges in building custom chips, and the effectiveness of domain-specific small models. The conversation also explores the transition towards a hybrid learning approach, regulating generative models, and Naveen's transformative journey from computer architecture to AI innovation.

44:26

Creator website

Episode guests

Naveen Rao

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Evolution towards transformer architectures in AI chips for specific workloads.

Shift from supervised learning to self-supervised learning for domain-specific data adaptation in LLMs.

Deep dives

Potential of Agentic Things in Future Technology

The podcast episode discusses the potential advancements in AI over the next few decades, envisioning technology that can replicate agency by formulating hypotheses, executing actions, observing consequences, and adapting based on results. Although energy requirements may be high initially, the progress is viewed optimistically, with the aim of fundamentally changing the world.

Bespoke Models vs. Dominant Use Cases

01:17

NVIDIA: The Gold Standard in Hardware Platforms

01:09

Introduction

4min

Evolution of Language Models and Custom Chip Building

15min

Specialized Small Models vs. GPT-4

4min

The Evolution towards a Hybrid Learning Approach

8min

Navigating Challenges in Regulating Generative Models

7min

Journey to AI Pioneering

6min

Naveen Rao, vice president of generative AI at Databricks, joins a16z's Matt Bornstein and Derrick Harris to discuss enterprise usage of LLMs and generative AI. Naveen is particularly knowledgeable about the space, having spent years building AI chips first at Qualcomm and then as the founder of AI chip startup Nervana Systems back in 2014. Intel acquired Nervana in 2016.

After a stint at Intel, Rao re-emerged with MosaicML in 2021. This time, he focused on the software side of things, helping customers train their own LLMs, and also fine-tune foundation models, on top of an optimized tech stack. Databricks acquired Mosaic in July of 2023.

This discussion covers the gamut of generative AI topics — from basic theory to specialized chips — to although we focus on how the enterprise LLM market is shaping up. Naveen also shares his thoughts on why he prefers finally being part of the technology in-crowd, even if it means he can’t escape talking about AI outside of work.

More information:

LLMs at Databricks

Mosaic Research