Pioneers of AI With Groq, Jonathan Ross is taking AI inference to new speeds
13 snips
Apr 9, 2025 
Jonathan Ross, Founder of Groq and former Google TPU pioneer, discusses the rapid evolution of AI chips capable of fast inference. He emphasizes that quicker processing not only enhances user experience but also slashes energy and compute costs. The conversation covers how Groq is shaking up the chip market and the global dynamics of AI inference. Ross highlights the emerging role of prompt engineering in AI and its potential to democratize access to technology while raising concerns about data safety.
AI Snips
Chapters
Transcript
Episode notes
AlphaGo and Compute
- Jonathan Ross helped optimize AlphaGo on Google's TPU, significantly improving its performance.
- This highlighted the importance of compute in AI quality.
Compute and Iteration
- Faster compute allows for more iterations, improving AI's quality, similar to refining an essay.
- This focus on latency is crucial, especially with reasoning models.
Grok vs. NVIDIA
- NVIDIA excels at training, which requires brute-force compute.
- Groq focuses on inference, which prioritizes cost, scale, and latency.