The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Managing Research Needs at the University of Michigan using Kubernetes w/ Bob Killen - #344
12 snips
Feb 3, 2020 Bob Killen, Research Cloud Administrator at the University of Michigan, shares insights on deploying Kubernetes to enhance research capabilities. He discusses how Kubernetes is transforming user experiences in diverse research areas and supports tools like Jupyter notebooks. Bob addresses concerns about balancing ML/AI needs amid broader usage and explores the challenges of managing long-running AI workloads. The conversation highlights the ongoing evolution of Kubernetes to support various applications, including collaborative efforts to improve usability.
AI Snips
Chapters
Transcript
Episode notes
Early Container Orchestration
- Bob Killen first used containers in 2015 for clinical workflows, before Docker's popularity.
- Initially, they considered Kubernetes but opted for Mesos due to Kubernetes' immaturity.
Shifting User Expectations
- Researchers are shifting from traditional HPC (logging into an SSH node and queuing jobs).
- They now expect Jupyter notebooks or science gateways for easier workload management.
Abstractions for Distributed Computing
- Distributed environments like Kubernetes require users to understand distributed computing.
- Abstractions like Argo workflows simplify this for some researchers, while others prefer direct control.