Eye On A.I. #281 Leon Song: The Research Driving Next-Gen Open-Source Models (Together AI)
Aug 25, 2025
Leon Song, VP of Research at Together AI, previously led AI initiatives at Microsoft. He sheds light on the rapid evolution of open-source AI models, explaining breakthroughs in speculative decoding and how they enhance performance. The conversation touches on the competitive landscape between open-source and proprietary systems, highlighting innovations like DeepSeq R1 and Llama 4. Leon also discusses empowering enterprises with data sovereignty through their cloud platform, sparking interest in the future of AI infrastructure.
AI Snips
Chapters
Transcript
Episode notes
Research-Driven Product Strategy
- Together AI pairs research (e.g., FlashAttention, RedPajama) with product-focused engineering to drive open-source model adoption.
- This research-first approach underpins their platform optimizations and customer-facing features.
Scaling Protein Structure Training
- At Microsoft Leon helped scale protein-structure training by reducing memory use and enabling three-times-longer sequences.
- That work enabled the open-source community and startups to train larger biology models effectively.
How Speculative Decoding Boosts Inference
- Speculative decoding uses a small speculator model and a larger verifier to speed inference without losing accuracy.
- High acceptance rates for the speculator are critical to realize meaningful latency gains.