20VC: AI Scaling Myths: More Compute is not the Answer | The Core Bottlenecks in AI Today: Data, Algorithms and Compute | The Future of Models: Open vs Closed, Small vs Large with Arvind Narayanan, Professor of Computer Science @ Princeton
Arvind Narayanan, a Princeton professor and co-author of "AI Snake Oil," challenges the myth that simply adding more compute equates to better AI performance. He emphasizes that data quality, not just volume, is crucial for advancements in AI. The conversation dives into the future of AI models, debating whether we'll have a few large dominant models or many specialized ones. Narayanan critiques current generative AI pitfalls and stresses the importance of genuine user experiences over misleading benchmark scores. His insights offer a fresh perspective on AI's evolving landscape.
51:55
forum Ask episode
web_stories AI Snips
view_agenda Chapters
menu_book Books
auto_awesome Transcript
info_circle Episode notes
insights INSIGHT
Compute Limits
Arvind Narayanan disagrees with the notion that more compute automatically leads to better AI models.
Data limitations and diminishing returns on compute are becoming significant bottlenecks.
insights INSIGHT
Data Bottleneck
YouTube's video data, while vast, is not significantly larger than current text data used to train large models.
This limits the potential for new emergent capabilities in text-based models.
insights INSIGHT
Synthetic Data Limits
Synthetic data helps improve existing data quality, not quantity.
Generating massive synthetic datasets is like a "snake eating its own tail," not adding new knowledge.
Get the Snipd Podcast app to discover more snips from this episode
What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
Sayash Kapoor
Arvind Narayanan
AI Snake Oil cuts through the confusion surrounding AI by explaining how it works, where it might be useful or harmful, and when companies are using AI hype to sell ineffective products. The book acknowledges the potential of some AI, like ChatGPT, while uncovering rampant misleading claims and describing serious harms in areas such as education, medicine, hiring, banking, insurance, and criminal justice. It explains the differences between types of AI, why organizations fall for AI snake oil, and warns of the dangers of unaccountable big tech companies controlling AI.
Arvind Narayanan is a professor of Computer Science at Princeton and the director of the Center for Information Technology Policy. He is a co-author of the book AI Snake Oil and a big proponent of the AI scaling myths around the importance of just adding more compute. He is also the lead author of a textbook on the computer science of cryptocurrencies which has been used in over 150 courses around the world, and an accompanying Coursera course that has had over 700,000 learners.
In Today's Episode with Arvind Narayanan We Discuss:
1. Compute, Data, Algorithms: What is the Bottleneck:
Why does Arvind disagree with the commonly held notion that more compute will result in an equal and continuous level of model performance improvement?
Will we continue to see players move into the compute layer in the need to internalise the margin? What does that mean for Nvidia?
Why does Arvind not believe that data is the bottleneck? How does Arvind analyse the future of synthetic data? Where is it useful? Where is it not?
2. The Future of Models:
Does Arvind agree that this is the fastest commoditization of a technology he has seen?
How does Arvind analyse the future of the model landscape? Will we see a world of few very large models or a world of many unbundled and verticalised models?
Where does Arvind believe the most value will accrue in the model layer?
Is it possible for smaller companies or university research institutions to even play in the model space given the intense cash needed to fund model development?
3. Education, Healthcare and Misinformation: When AI Goes Wrong:
What are the single biggest dangers that AI poses to society today?
To what extent does Arvind believe misinformation through generative AI is going to be a massive problem in democracies and misinformation?
How does Arvind analyse AI impacting the future of education? What does he believe everyone gets wrong about AI and education?
Does Arvind agree that AI will be able to put a doctor in everyone's pocket? Where does he believe this theory is weak and falls down?
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch 
