The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch 20VC: AI Scaling Myths: More Compute is not the Answer | The Core Bottlenecks in AI Today: Data, Algorithms and Compute | The Future of Models: Open vs Closed, Small vs Large with Arvind Narayanan, Professor of Computer Science @ Princeton
67 snips
Aug 28, 2024 
Arvind Narayanan, a Princeton professor and co-author of "AI Snake Oil," challenges the myth that simply adding more compute equates to better AI performance. He emphasizes that data quality, not just volume, is crucial for advancements in AI. The conversation dives into the future of AI models, debating whether we'll have a few large dominant models or many specialized ones. Narayanan critiques current generative AI pitfalls and stresses the importance of genuine user experiences over misleading benchmark scores. His insights offer a fresh perspective on AI's evolving landscape.
AI Snips
Chapters
Books
Transcript
Episode notes
Compute Limits
- Arvind Narayanan disagrees with the notion that more compute automatically leads to better AI models.
- Data limitations and diminishing returns on compute are becoming significant bottlenecks.
Data Bottleneck
- YouTube's video data, while vast, is not significantly larger than current text data used to train large models.
- This limits the potential for new emergent capabilities in text-based models.
Synthetic Data Limits
- Synthetic data helps improve existing data quality, not quantity.
- Generating massive synthetic datasets is like a "snake eating its own tail," not adding new knowledge.
