20VC: Scale's Alex Wang on Why Data Not Compute is the Bottleneck to Foundation Model Performance, Why AI is the Greatest Military Asset Ever, Is China Really Two Years Behind the US in AI and Why the CCPs Industrial Approach is Better than Anyone Else's
Jun 12, 2024
auto_awesome
Scale.ai's CEO Alex Wang discusses improving AI model performance with better data, the military potential of AI, comparing China's advancements, and the CCP's industrial approach.
Data is crucial for AI model performance, surpassing compute limitations.
AI could be a potent military asset, potentially exceeding nuclear weapons.
CCP's industrial approach excels in driving technological advancements and military strategies.
Deep dives
AI Technology as a Military Asset
AI technology is considered a potent military asset, potentially surpassing nuclear weapons in military significance. The scenario where countries like China or Russia possess artificial general intelligence (AGI) while others lack it raises concerns about conquest possibilities. Centralized systems like the CCP in China excel in driving critical industries aggressively, showcasing a potential advantage in technological advancement and military strategies.
Challenges in AI Model Development
The podcast delves into the current challenges in AI model development, focusing on the concept of diminishing returns amidst increasing computational capabilities. Despite significant investments in computational power post-GPT-4, no major breakthroughs surpassing GPT-4 have emerged, leading to a reflection on the trio of pillars - compute, data, and algorithms. The discussion highlights the importance of addressing the limitation around data availability and its essential role in pushing AI model capabilities forward.
Transition to Data Abundance for AI Progress
The critical shift from data scarcity to data abundance for AI advancement is emphasized, pointing towards the necessity of frontier data to enhance AI model capabilities. The podcast explores the concept of frontier data as a crucial component, involving complex reasoning chains and diverse data sources. Strategies such as mining existing enterprise data and human-AI collaboration in data creation are proposed to propel AI models towards enhanced performance and innovative problem-solving abilities.
Impact of AI Technology on Military Assets
AI technology is discussed as a significant military asset, potentially surpassing even nuclear capabilities. The scenario of one country having AGI compared to another without it is analyzed, highlighting AGI's strategic advantage in warfare. The importance of geopolitical tensions and the need for Western nations to prevent adversaries from leveraging AI for conquest are emphasized. The debate between open systems and closed systems in AI development for military reasons is deliberated, suggesting a nuanced approach to balancing advancement and security.
Challenges and Strategies in Company Building Principles
Company-building principles and scaling challenges are examined, emphasizing the need for quality over hypergrowth. The complexities of team expansion and maintaining excellence during rapid growth are explored. The narrative delves into the importance of hiring top talent ('Navy SEALs') to ensure high standards within a growing organization. Lessons learned from past management ‘fuck-ups,’ such as maintaining a high bar during rapid scaling, reflect on the critical balance between expansion and sustaining excellence for long-term success.
Alex Wang is the Founder and CEO @ Scale.ai, the company that allows you to make the best models with the best data. To date, Alex has raised $1.6BN for the company with a last reported valuation of $14BN earlier this year. Scale tripled their ARR in 2023 and is expected to hit $1.4BN in ARR by the end of 2024. Their investors include Accel, Index, Thrive, Founders Fund, Meta and Nvidia to name a few.
In Today's Show with Alex Wang We Discuss:
1. Foundation Models: Diminishing Returns:
What are the three core pillars that can meaningfully improve foundation models performance?
Why is data the single largest bottleneck to the performance of models today?
What data do we need to capture that we do not currently, that will have the biggest impact on model performance moving forward?
Will we see the largest companies in the world revert back to on-prem with the increasing security challenges of migrating all customer data to foundation models?
2. AI: A Military Asset in Global Conflict: China + Russia
Why does Alex believe that AI has the potential to be an even more powerful military asset than nuclear weapons?
If this is the case, should we have open systems? Do we not have to have closed systems?
Why does Alex believe that the CCP's approach to industrial policy is better than anyone else's?
How does Alex evaluate the rise of Chinese EV car manufacturers in the last few years?
Does Alex really believe that China is two years behind the US in the AI race?
3. "I Get Fairer Treatment in Congress than in the Press":
Why does Alex believe that the best PR is no PR?
Why does Alex believe that he got fairer treatment in congress than he does in the media?
Why does Alex believe that all founders should look to own their own distribution channels today?
4. Alex Wang: AMA:
What are some of Alex's biggest lessons from Patrick Collison on the impact that a hot company brand has on the ability for that company to hire the best?
Does Alex think Trump is going to win? What would be the impact if he were to?
Why does Alex believe that enterprise software will be changed forever in the next few years?
What question is Alex never asked that he thinks he should be asked?
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode