Chris Miller, author of Chip War, discusses the rise of Nvidia and their role in generative AI. The episode also explores how big tech companies are trying to fend off AI startups. They discuss the importance of developer relations, Facebook's role in democratizing AI, concerns about open source models, attitudes towards open source, and the significance of chips in AI and the concentration of chip production.
NVIDIA's GPUs have become the go-to choice for training cutting-edge AI models.
NVIDIA effectively plays the role of a kingmaker in the industry by supplying their highly sought-after AI chips.
There is a growing exploration of performing inference on the edge using devices like phones or cars, potentially reducing latency.
Deep dives
NVIDIA's Evolution from Gaming Chips to AI
NVIDIA, initially focused on graphics chips for gaming, has evolved into a key player in the AI industry. Over a decade ago, the company recognized the potential of AI and began investing in building a software ecosystem around their chips. Today, NVIDIA is the dominant player in the market, with an estimated 90% market share for cutting-edge AI systems.
The Importance of GPUs in AI
GPUs (graphics processing units) play a crucial role in AI tasks due to their parallel processing capabilities. AI tasks require massive amounts of data and computations, and GPUs excel at processing this data in parallel, making them much faster than traditional CPUs. NVIDIA's GPUs have become the go-to choice for training cutting-edge AI models.
Scarcity of NVIDIA Chips and Kingmaker Status
NVIDIA's AI chips, especially the advanced H100s, are currently in high demand and scarce supply. NVIDIA effectively plays the role of a kingmaker in the industry, as their chips are sought after by startups and other companies to run AI-related tasks. NVIDIA's partnerships with cloud computing providers and efforts to build a fragmented market further strengthen their position in the AI chip industry.
Competition and Rivalry in the AI Chip Market
While there are competitors like Google and AMD in the AI chip market, NVIDIA currently holds a significant advantage. Google has developed its own chip called the TPU (Tensor Processing Unit), but many companies still prefer NVIDIA's GPUs. Intel is also trying to compete but currently relies on TSMC, a Taiwanese manufacturer, for advanced chip production. NVIDIA's ecosystem and established position make it difficult for competitors to gain significant market share.
AI infrastructure and the need for compute power
The demand for compute power in AI training is so large that there is a deficit. Firms like OpenAI need all of their infrastructure and even more, but sharing is not feasible due to the high demand and limited access to compute power.
The role of inference on the edge and the potential of Apple
While AI training mainly occurs in big data centers, there is a growing question about the future of inference. Inference on the edge, where computing happens in devices like phones or cars, is being explored as an alternative to data center-based inference. Companies like Apple could play a significant role in this area with their chips and devices, as they have the potential to bring inference closer to the user and reduce latency.
For this week’s episode, I spoke with Chris Miller, the author of Chip War, about the rise of Nvidia.
While OpenAI gets the lion’s share of the public adulation for the sudden excitement about generative intelligence, Nvidia’s H100 chips are powering much of the generative AI frenzy. Nvidia’s stock has climbed over 200% over the past 12 months. And the company has become a key investor in generative AI startups.
Miller (who comes on the show around the 41-minute mark) talks through Nvidia’s history and the geopolitical war raging over the production of chips.
In the first part of the episode, Cerebral Valley AI Summit co-hosts Max Child, James Wilsterman, and Idiscuss how big technology companies are working to fend off this new generation of AI startups.
