Machine Learning Street Talk (MLST)

Why Your GPUs are underutilised for AI - CentML CEO Explains

28 snips
Nov 13, 2024
Gennady Pekhimenko, CEO of CentML and associate professor at the University of Toronto, dives into the intricacies of AI system optimization. He illuminates the challenges of GPU utilization, revealing why many companies only harness 10% efficiency. The conversation also touches on 'dark silicon,' the competition between open-source and proprietary AI, and the need for strategic refinement in enterprise AI infrastructure. Pekhimenko's insights blend technical depth with practical advice for enhancing machine learning applications in modern businesses.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
INSIGHT

Open-Source Models Closing Gap

  • Open-source models are rapidly improving and closing the gap with proprietary models.
  • This is beneficial for wider access and innovation in the AI space.
INSIGHT

Transformer Models Still Dominant

  • Attention-based transformer models remain dominant in AI architecture, with no clear replacement yet.
  • Future development will likely focus on building systems on top of these models, not replacing them.
ANECDOTE

Low GPU Utilization

  • Gennady Pekhimenko observed only 10% GPU utilization in early ML workloads at Microsoft Research.
  • This highlighted the gap in understanding compute between ML and systems communities.
Get the Snipd Podcast app to discover more snips from this episode
Get the app