AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
LessWrong (Curated & Popular)
“What o3 Becomes by 2028” by Vladimir_Nesov
Jan 9, 2025
Vladimir Nesov, author known for his insights on AI, dives deep into the future of OpenAI's O3 training systems. He discusses the significance of upcoming investments in scaling AI capabilities with models projected to train at unprecedented FLOP levels. The conversation highlights the balance between data quality and quantity, emphasizing the need for 50 trillion training tokens. Nesov also evaluates the current state of GPT-4 and its competitors, pondering what advancements might emerge by 2028 in this rapidly evolving landscape.
08:40
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- By 2028, OpenAI's O3 is expected to drive advancements in AI training systems with capabilities reaching 5x10^28 FLOPs amidst significant funding support.
- The race for AI scalability is intensifying as companies invest in massive data centers to enhance computational efficiency and model performance.
Deep dives
Future Training Systems and Performance
By 2028, significant advancements in training systems are anticipated, with funding now more substantial for developing technologies that can support immense computational demands. OpenAI's O3 may facilitate pre-training models capable of reaching 5x10^28 flops, while the constraints of natural text data will not hinder progress until much larger flops are achieved. This scaling of pre-training is expected to remain robust, demonstrating that leading models like GPT-4 set a benchmark for future developments despite others catching up. The race to enhance these systems is intensifying, with expected outputs surpassing current capabilities as hardware continues to evolve and expand.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode