Yuan Tang is a principal software engineer at Red Hat, focusing on OpenShift AI, and is a leader in Kubernetes WG Serving. Eduardo Arango, a software engineer at NVIDIA, specializes in making Kubernetes suitable for high-performance computing. They delve into the challenges of AI model serving, discussing startup times and Kubernetes API limitations. The conversation also covers orchestration complexities for large language models and highlights innovative solutions like Model Mesh to optimize multi-host environments. Engagement and collaboration in Kubernetes working groups are urged for community-driven advancements.
