Eye On A.I.

#228 Rodrigo Liang: How SambaNova Systems Is Disrupting AI Inference

Jan 1, 2025

Rodrigo Liang, co-founder and CEO of SambaNova Systems and an expert in high-performance chip design, dives deep into the world of AI inference technology. He shares SambaNova's groundbreaking record-breaking models that deliver incredible speed and efficiency. The discussion highlights the shift from AI training to real-time applications and the importance of power efficiency. Rodrigo also explores the competitive landscape against giants like NVIDIA, advocating for a new era of scalable, user-friendly AI solutions that empower enterprises without costly infrastructure.

23:54

Creator website

Episode guests

Rodrigo Liang

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Rodrigo Liang discusses how SambaNova Systems is disrupting the AI inference landscape through innovative, power-efficient chip designs and models.

The podcast highlights the shift from AI training to inference, emphasizing the growing demand for speed, real-time processing, and operational efficiency in enterprises.

Deep dives

The Shift in AI Inference Landscape

NVIDIA's previous dominance in AI training is shifting as competitors emerge in the inference space. Many developers are exploring alternative offerings that provide APIs across various platforms, making it easier to access tools and services without reliance on CUDA. The growing number of developers opting for options like Google Cloud and AWS indicates a significant change in the competitive landscape. This transition is driven by the need for better power performance and cost-effectiveness, which other companies can now offer alongside NVIDIA.

Intro

3min

Innovations in AI Inference Technology

4min

Disrupting AI Inference: SambaNova's Approach to Cloud Adoption

2min

Advancements in AI Inference Technology

12min

Navigating the Competitive Landscape of AI Hardware

3min

This episode is sponsored by RapidSOS. Close the safety gap and transform your emergency response with RapidSOS.

Visit https://rapidsos.com/eyeonai/ today to learn how AI-powered safety can protect your people and boost your bottom line.

In this episode of the Eye on AI podcast, we explore the world of AI inference technology with Rodrigo Liang, co-founder and CEO of SambaNova Systems.

Rodrigo shares his journey from high-performance chip design to building SambaNova, a company revolutionizing how enterprises leverage AI through scalable, power-efficient solutions. We dive into SambaNova’s groundbreaking achievements, including their record-breaking inference models, the Lama 405B and 70B, which deliver unparalleled speed and accuracy—all on a single rack consuming less than 10 kilowatts of power.

Throughout the conversation, Rodrigo highlights the seismic shift from AI training to inference, explaining why production AI is now about speed, efficiency, and real-time applications. He details SambaNova’s approach to open-source models, modular deployment, and multi-tenancy, enabling enterprises to scale AI without costly infrastructure overhauls.

We also discuss the competitive landscape of AI hardware, the challenges of NVIDIA’s dominance, and how SambaNova is paving the way for a new era of AI innovation. Rodrigo explains the critical importance of power efficiency and how SambaNova’s technology is unlocking opportunities for enterprises to deploy private, secure AI systems on-premises and in the cloud.

Discover how SambaNova is redefining AI for enterprise adoption, enabling real-time AI, and setting new standards in efficiency and scalability.

Don’t forget to like, subscribe, and hit the notification bell to stay updated on the latest breakthroughs in AI, technology, and enterprise innovation!

Stay Updated:

Craig Smith Twitter: https://twitter.com/craigss

Eye on A.I. Twitter: https://twitter.com/EyeOn_AI