Infinite Curiosity Pod with Prateek Joshi AI Infra for Long Context Model Training | Anna Patterson, founder of Ceramic AI
Jun 17, 2025
In this conversation, Anna Patterson, cofounder of Ceramic AI and former VP Engineering at Google, shares her insights on AI infrastructure for model training. She discusses achieving a 2.5x speed-up in long context training and the nuances between short, medium, and long contexts. Anna also dives into the importance of differentiating good data from bad, particularly in complex domains. She reflects on the significance of synthetic data and recent AI advancements, offering a glimpse into the future of personalized AI models.
AI Snips
Chapters
Transcript
Episode notes
Scaling Efficiency Is Key Challenge
- The biggest challenge in AI infrastructure is scaling up efficiently across thousands of machines.
- Achieving speed comparable to small setups at large scale is key for effective training.
Inference Costs Rising Fast
- In AI budgets, training used to dominate but inference costs are now rising faster with user traffic.
- Inference could soon be a larger cost factor than training in large models.
AI Infrastructure Layers Overview
- AI infra layers range from GPUs in data centers to software managing health and training.
- The ecosystem includes both hardware providers and model developers like OpenAI and Meta.