The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch 20VC: Scale's Alex Wang on Why Data Not Compute is the Bottleneck to Foundation Model Performance, Why AI is the Greatest Military Asset Ever, Is China Really Two Years Behind the US in AI and Why the CCPs Industrial Approach is Better than Anyone Else's
62 snips
Jun 12, 2024 
Alex Wang, Founder and CEO of Scale AI, delves into the critical role data plays in AI's future performance. He argues that data, not compute, is the main bottleneck for foundation models. The discussion pivots to AI as a formidable military asset, potentially surpassing nuclear weapons, and examines China's approach to industrial policy in AI development. Wang challenges the notion that China lags behind the U.S. in AI, exploring innovations in the Chinese EV market and the implications of these geopolitical dynamics.
AI Snips
Chapters
Transcript
Episode notes
Data Bottleneck
- Compute is not the only factor limiting foundation model performance.
- Data limitations are a significant bottleneck to improvement.
Data Wall
- The current plateau in model performance stems from hitting a "data wall".
- Algorithmic improvements and increased computational capability are insufficient without more data.
Frontier Data
- Existing data is not enough for models to go beyond emulating the internet.
- We need "frontier data" that captures complex reasoning and thought processes currently missing from the internet.