The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch 20VC: OpenAI's Newest Board Member, Zico Colter on The Biggest Bottlenecks to the Performance of Foundation Models | The Biggest Questions and Concerns in AI Safety | How to Regulate an AI-Centric World
10 snips
Sep 4, 2024 
Zico Colter, a professor and the director of the Machine Learning Department at Carnegie Mellon University, discusses AI's biggest bottlenecks. He delves into data utilization, the diminishing returns of compute power, and looming algorithmic challenges. Zico critiques prevalent concerns about AI safety, urging listeners to focus on overlooked risks while navigating an AI-centric world. The conversation touches on the transformative role of AI technology and the urgent need for effective regulation to ensure alignment with human interests.
AI Snips
Chapters
Transcript
Episode notes
LLM Mechanics
- LLMs predict words in sequence, trained on internet data.
- This simple process generating intelligence is a remarkable scientific discovery.
Data Scarcity?
- Easily available, high-quality internet text data has been largely used.
- However, much more data (internal, multimodal) remains untapped due to compute limits.
Multimodal Challenges
- Compute limits hinder multimodal data use, despite its potential value.
- Text data represents a distilled form of information, unlike video or audio.