DataTalks.Club

Redefining AI Infrastructure: Open-Source, Chips, and the Future Beyond Kubernetes – Andrey Cheptsov

Jan 31, 2025

56:55

In this podcast episode, we talked with Andrey Cheptsov about The future of AI infrastructure.

About the Speaker:

Andrey Cheptsov is the founder and CEO of dstack, an open-source alternative to Kubernetes and Slurm, built to simplify the orchestration of AI infrastructure. Before dstack, Andrey worked at JetBrains for over a decade helping different teams make the best developer tools.

During the event, the guest, Andrey Cheptsov, founder and CEO of dstack, discussed the complexities of AI infrastructure. We explore topics like the challenges of using Kubernetes for AI workloads, the need to rethink container orchestration, and the future of hybrid and cloud-only infrastructures. Andrey also shares insights into the role of on-premise and bare-metal solutions, edge computing, and federated learning.

00:00 Andrey's Career Journey: From JetBrains to DStack

5:00 The Motivation Behind DStack

7:00 Challenges in Machine Learning Infrastructure

10:00 Transitioning from Cloud to On-Prem Solutions

14:30 Reflections on OpenAI's Evolution

17:30 Open Source vs Proprietary Models: A Balanced Perspective

21:01 Monolithic vs. Decentralized AI businesses

22:05 The role of privacy and control in AI for industries like banking and healthcare

30:00 Challenges in training large AI models: GPUs and distributed systems

37:03 DeepSpeed's efficient training approach vs. brute force methods

39:00 Challenges for small and medium businesses: hosting and fine-tuning models

47:01 Managing Kubernetes challenges for AI teams