Grit Synthetic Data and the Future of AI | Cohere CEO Aidan Gomez
23 snips
Nov 17, 2025 
Aidan Gomez, co-founder and CEO of Cohere and a key figure behind the 'Attention Is All You Need' paper, dives into the transformation of synthetic data from a niche concept to an essential tool for AI. He discusses the importance of efficiency in model development and how expectations of continuous scaling are being reassessed. Aidan also critiques alarmist narratives around AGI and highlights the substantial enterprise opportunities currently untapped. His insights emphasize the role of AI in boosting productivity and driving economic growth.
AI Snips
Chapters
Transcript
Episode notes
Efficiency Fueled The Transformer
- The Transformer succeeded because it prioritized computational efficiency and GPU-friendly design.
- Models that scale well dominate as training shifted to massive multi-GPU setups.
Submitting The Transformer Paper
- Aidan recounts sleeping at the office the night they submitted the Transformer paper to NeurIPS as an intern.
- He remembers colleagues predicting the paper would be a big deal, well beyond initial expectations.
Scaling Gains Are Tapering
- Scaling delivered much longer gains than many expected but is now showing diminishing returns in pre-training.
- Labs are shifting focus to better data, training methods, and targeted capabilities rather than raw scale alone.