The Shift from Training to Inference in AI

Steeve Morin is the Founder & CEO @ ZML, a next-generation inference engine enabling peak performance on a wide range of chips. Prior to founding ZML, Steeve was the VP Engineering at Zenly for 7 years leading eng to millions of users and an acquisition by Snap. 

In Today’s Episode We Discuss:

04:17 How Will Inference Change and Evolve Over the Next 5 Years

09:17 Challenges and Innovations in AI Hardware

15:38 The Economics of AI Compute

18:01 Training vs. Inference: Infrastructure Needs

25:08 The Future of AI Chips and Market Dynamics

34:43 Nvidia's Market Position and Competitors

38:18 Challenges of Incremental Gains in the Market

39:12 The Zero Buy-In Strategy

39:34 Switching Between Compute Providers

40:40 The Importance of a Top-Down Strategy for Microsoft and Google

41:42 Microsoft's Strategy with AMD

45:50 Data Center Investments and Training

46:40 How to Succeed in AI: The Triangle of Products, Data, and Compute

48:25 Scaling Laws and Model Efficiency

49:52 Future of AI Models and Architectures

57:08 Retrieval Augmented Generation (RAG)

01:00:52 Why OpenAI’s Position is Not as Strong as People Think

01:06:47 Challenges in AI Hardware Supply