The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Rethinking Pre-Training for Agentic AI with Aakanksha Chowdhery - #759
93 snips
Dec 17, 2025 Aakanksha Chowdhery, a machine learning researcher from Reflection, dives into the future of agentic AI. She critiques the current reliance on post-training techniques, advocating for a transformative approach to pre-training. Aakanksha highlights the need for evolving attention mechanisms and tailored training data to enhance long-term reasoning and planning. She discusses the importance of 'trajectory' training data and the risks of synthetic data, all while underscoring the significance of rigorous evaluation in building agentic models.
AI Snips
Chapters
Transcript
Episode notes
Pre-Training Needs Rethink For Agents
- Pre-training evaluated on static benchmarks limits models for agentic tasks that interact with environments.
- Rethinking pre-training fundamentals is necessary to enable interactive, multi-step agent behaviors.
Scaling Teaches Cross-Stack Lessons
- Aakanksha led infrastructure work for PaLM and early Gemini models and learned how scale magnifies issues.
- She emphasizes cross-stack thinking when pre-training models that run for months.
Post-Training Alone Is Limiting
- Post-training tweaks can help but they fundamentally limit achievable capabilities.
- Core capabilities like planning, long-context reasoning, and tool learning require changes earlier in training.