Optimizing Inference Load Balancing in Multi-Cluster Environments

Guests are Nick Eberts and Jon Li. Nick is a Product Manager at Google working on Fleets and Multi-Cluster and Jon is a Software Engineer at Google working on AI Inference on Kubernetes. We discussed the newly announced Multi Cluster Orchestrator (MCO) and the challenges of running multiple clusters.

Do you have something cool to share? Some questions? Let us know:

- web: kubernetespodcast.com

- mail: kubernetespodcast@google.com

- twitter: @kubernetespod

- bluesky: @kubernetespodcast.com

• Etcd has released version 3.6.0

• Kubernetes 1.33 is now available in the Rapid channel in GKE

• Kyverno 1.14.0 was released

• Nick Eberts on LinkedIn

• Jon Li on LinkedIn

• MCO Blog

• MCO Repo

• Cluster Inventory API

• ClusterProfile API

• Gemma 3 vLLM Sample (deploy on Google Cloud using Terraform and Argo CD)

• Hello World Sample (deploy on Google Cloud using Terraform and Argo CD)

• Gateway API Inference Extension