Naveen Rao, VP of generative AI at Databricks, discusses enterprise LLMs and generative AI, highlighting the evolution of language models, challenges in building custom chips, and the effectiveness of domain-specific small models. The conversation also explores the transition towards a hybrid learning approach, regulating generative models, and Naveen's transformative journey from computer architecture to AI innovation.
Evolution towards transformer architectures in AI chips for specific workloads.
Shift from supervised learning to self-supervised learning for domain-specific data adaptation in LLMs.
Deep dives
Potential of Agentic Things in Future Technology
The podcast episode discusses the potential advancements in AI over the next few decades, envisioning technology that can replicate agency by formulating hypotheses, executing actions, observing consequences, and adapting based on results. Although energy requirements may be high initially, the progress is viewed optimistically, with the aim of fundamentally changing the world.
Evolution of Enterprise LLM Market and NVIDIA's Dominance
The discussion delves into the evolution of the enterprise Large Language Model (LLM) market, focusing on the impact of models like GPT-4. It touches on NVIDIA's continued dominance in AI workloads due to their ability to identify trends and execute efficiently, posing challenges for competitors in terms of hardware lock-ins.
Custom Hardware Challenges and Transformation in AI
The conversation delves into the challenges faced in building custom AI chips when supporting various neural network families. The episode highlights the shift towards transformer architectures and the opportunities for chip companies to tailor products for specific workloads, suggesting that modifications and new paradigms may be needed beyond the transformer architecture.
Transition from Supervised Learning to Self-Supervised Learning Era
The episode explores the shift from supervised learning to self-supervised learning, emphasizing the importance of customized reasoning and the ability of large language models to adapt to domain-specific data. It discusses the role of pre-training and fine-tuning in creating models that can outperform standardized models like GPT-4. The emphasis is on the evolution towards a more practical and economically feasible AI ecosystem.
Naveen Rao, vice president of generative AI at Databricks, joins a16z's Matt Bornstein and Derrick Harris to discuss enterprise usage of LLMs and generative AI. Naveen is particularly knowledgeable about the space, having spent years building AI chips first at Qualcomm and then as the founder of AI chip startup Nervana Systems back in 2014. Intel acquired Nervana in 2016.
After a stint at Intel, Rao re-emerged with MosaicML in 2021. This time, he focused on the software side of things, helping customers train their own LLMs, and also fine-tune foundation models, on top of an optimized tech stack. Databricks acquired Mosaic in July of 2023.
This discussion covers the gamut of generative AI topics — from basic theory to specialized chips — to although we focus on how the enterprise LLM market is shaping up. Naveen also shares his thoughts on why he prefers finally being part of the technology in-crowd, even if it means he can’t escape talking about AI outside of work.