Demetrios and Mohan Atreya break down the GPU madness behind AI — from supply headaches and sky-high prices to the rise of nimble GPU clouds trying to outsmart the giants. They cover power-hungry hardware, failed experiments, and how new cloud models are shaking things up with smarter provisioning, tokenized access, and a whole lotta hustle. It's a wild ride through the guts of AI infrastructure — fun, fast, and full of sparks!
Big thanks to the folks at Rafay for backing this episode — appreciate the support in making these conversations happen!
// Bio
Mohan is a seasoned and innovative product leader currently serving as the Chief Product Officer at Rafay Systems. He has led multi-site teams and driven product strategy at companies like Okta, Neustar, and McAfee.
// Related Links
Websites: https://rafay.co/
~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Mohan on LinkedIn: /mohanatreya
Timestamps:
[00:00] AI/ML Customer Challenges
[04:21] Dependency on Microsoft for Revenue
[09:08] Challenges of Hypothesis in AI/ML
[12:17] Neo Cloud Onboarding Challenges
[15:02] Elastic GPU Cloud Automation
[19:11] Dynamic GPU Inventory Management
[20:25] Terraform Lacks Inventory Awareness
[26:42] Onboarding and End-User Experience Strategies
[29:30] Optimizing Storage for Data Efficiency
[33:38] Pizza Analogy: User Preferences
[35:18] Token-Based GPU Cloud Monetization
[39:01] Empowering Citizen Scientists with AI
[42:31] Innovative CFO Chatbot Solutions
[47:09] Cloud Services Need Spectrum
