Odd Lots Jack Morris on Finding the Next Big AI Breakthrough
127 snips
Sep 26, 2025 
Jack Morris, an AI researcher and Ph.D. candidate at Cornell, delves into the unpredictable nature of AI breakthroughs. He discusses the intricate balance between open and closed research labs, revealing how proprietary data can set teams apart. Jack also explores the complexities of measuring model performance with benchmarks and the distinction between supervised and reinforcement learning. With thoughts on the future of personalization and the challenges of monetizing curiosity-driven research, he offers a fresh perspective on the evolving AI landscape.
AI Snips
Chapters
Transcript
Episode notes
Research Spans The Whole Stack
- AI research spans hardware, data, training algorithms, and evaluation, not just model architecture.
- Jack Morris studies how much information models actually store and how to measure it in bits.
Benchmarks Don't Equal Real-World Use
- Teams evaluate models with benchmarks like coding datasets and math competitions to show progress.
- Benchmarks can improve while real-world usefulness still feels misaligned.
ELO Ratings Capture Qualitative Gaps
- Human pairwise comparisons (ELO-style) capture qualitative differences models' metrics miss.
- ELO ladders help rank models on style and subjective quality beyond raw benchmarks.