Kubernetes Podcast from Google

We finally scheduled some time to talk to David Oppenheimer. David, a software engininer at Google, has been working on scheduling there since 2007, including on both Borg and Omega. That experience naturally led to him working on the Kubernetes scheduler, as well as starting SIG Scheduling. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Last week’s discussion about ice cream pies Vegemite ice cream, and a friendly reminder that New Zealand is not Australia Mutton ice cream is not a thing A bear in the kiddie pool News of the week Google Traffic Director supports proxyless gRPC New Relic open sources its agents Lyft drops the Clutch Conftest joins the Open Policy Agent project Emissary, from GitHub VS Code Docker extension can now run containers in Azure Container Instances Debugging Incidents in Google’s Distributed Systems by Beth Cooper and Charisma Chan Hashicorp Consul Service on Azure is GA Gloo Federation for gloo’ing your Gloos together with gloo The AWS EKS CIS ben chm ark Changes to Aqua Wave and Aqua Enterprise Snyk’s developer-first prioritization capabilities Carbonetes launch PR Prevasio launch PR DOMA: domain-oriented microservices architecture at Uber by Adam Gluck Links from the interview Papers co-written by David: Large-scale cluster management at Google with Borg Borg, Omega and Kubernetes SIG Scheduling WG Multi-Tenancy App Engine Interviews with David’s colleagues on Borg and Omega: Episode 22, with Dawn Chen Episode 43, with Brian Grant Episode 111, with Wojciech Tyczynski Omega features: The Omlet Pod disruption budgets Taints and Tolerations Optimistic concurrency control Scheduler features Predicates and priorities Labels and selectors Node affinity and anti-affinity Pod affinity and anti-affinity Pod priority and preemption Disruption budgets Taints and tolerations Two level scheduling Mesos optimistic offers Kubernetes scheduler in Bash Firmament and integration in Kubernetes via Poseidon Configuration tools kpt kustomize David Oppenheimer on Twitter

Released on the same day as Kubernetes, cadvisor is a container monitoring daemon that collects metrics and serves them to monitoring tools. It’s built into the Kubelet, and underpins many components in Kubernetes, such as eviction and autoscaling. David Ashpole of Google Cloud is TL of Kubernetes SIG Instrumentation, and the maintainer of cadvisor; he joins Adam and Craig this week to explain where instrumentation fits in the stack, and what you should do as a Kubernetes maintainer vs. a cluster administrator. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week In Craig’s neighbourhood: Books More books Some less popular items Masks Archie the Mammoth National Ice Cream Day Carmel Caramel News of the week GKE Ingress features: BackendConfig CRD Cloud CDN Backend service timeout Connection draining timeout HTTP access logging Identity-Aware Proxy (IAP) Session affinity User-defined request headers Cloud Armor security policies (Beta) FrontendConfig CRD (Beta) Custom GCLB health checks (Beta) SSL policies (Beta) Exposing services on GKE OpenShift 4.5 OKD4 Spring Cloud Data Flow for Kubernetes from VMware; part of the Spring Runtime package k8spin.cloud is closing and making their code open source Review of k8spin from launch Custom Pod Autoscaler (and docs) by Jamie Thompson Envoy 1.15 round-up from Tetrate; release notes from the team Fluent Bit 1.5 summary at the CNCF k3d v3.0 and new web site Best practices for creating a highly available GKE cluster Recommended alerts for AKS Ingress support added to AWS App Mesh Platform9 adds new apps to their Managed Kubernetes Service Episode 88, with Madhura Maskasky CVE-2020-8557: Node disk DOS by writing to container /etc/hosts CVE-2020-8559: Privilege escalation from compromised node to cluster Alcide write-up Threat Alert: Attacker Building Malicious Images Directly on Your Host from Aqua Security Certified Kubernetes Security Specialist (CKS) coming in November Sign up for a free pass to Virtual KubeCon EU keynotes Diving Into Istio 1.6 Certificate Rotation by Christian Posta Links from the interview SIG Instrumentation inodes Eviction on inodes cgroups cadvisor Launched on the same day as Kubernetes Monitoring metrics with Prometheus Victor Marmol and Vish Kannan Episode 22, with Dawn Chen CRI Resource metrics pipeline Heapster Metrics Server kube-state-metrics Managing Your Costs on Kubernetes by Karl Stoney from Autotrader Episode 52, with Russell Warman and Karl Stoney Metrics Stability Framework Structured logging Distributed tracing in Kubernetes Node out of memory eviction Pod priority David Ashpole on Twitter

An open source license grants rights on copyright and patents, but not trademarks. Chris DiBona has some ideas on how to address that. He has spent his career in open source, including over 15 years running Google’s Open Source Programs Office, and is one of the directors of the new Open Usage Commons. It launched last week with three projects - Angular, Gerrit and Istio - transferring their trademarks. Chris joins Adam and Craig to talk about Google’s work in open source, and why a new organisation is needed. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Software defined radio POGSAG The fuzz Talking to the International Space Station Breaker breaker News of the week SUSE to acquire Rancher Episode 57, with Darren Shepherd Open Usage Commons: OUC Board announcement Google announcement Istio blog post IBM opinion Governance updates Operator Framework and Contour accepted into the CNCF BigQuery Omni Kubernetes has caught up with YARN according to Datamechanics Kubernetes networking: why is this so dang hard? by Tim Hockin Episode 41 Announcing Kustomize support for Pulumi Cinderella clusters from Soluble Google’s Anthos comes to HPE Greenlake AWS: AWS partners with Docker Docker partners with AWS AWS Copilot for ECS cdk8s-plus AKS adds console RBAC and policy integration Kublr adds in-place upgrades and external clusters D2iQ want to teach you Links from the interview Chris DiBona VA Linux San Mehat Google Search Appliance Maintainer of Git Author of Git Ping pong balls on a bus AMP joined OpenJS Foundation and has now graduated WASM became a W3C standard Google Summer of Code Melange Open Usage Commons Apache Software License v2 and GPL v3 Open Source Definition Angular, Gerrit and Istio OUC board members Debian Free Software Guidelines Google Contributor License Agreement Apache Contributor License Agreement Developer Certificate of Origin Istio governance: Steering Committee and TOC Silicon Valley Chris’s IMDB page Palo Alto fiber ring (and today) Chris DiBona on Twitter Open Source at Google

Before Kubernetes was launched, it could have at most 25 nodes in a cluster. At 1.0, the target was 100. Meanwhile, Borg, Omega and Mesos were all running away at 10,000. What did it take to get Kubernetes to this number, and above? SIG Scalability and GKE Tech Lead Wojciech Tyczynski tells us. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Follow-up: Chairs, from Episode 107 Christmas trees, from Episode 104 Kids music The duck song The duck joke Autotune the News The duck song goes viral on TikTok Walmart Yodeling Kid News of the week KubeCon US goes virtual PromCon schedule AWS App2Container Episode 48, with Issy Ben-Shaul GKE brings Node Local DNS cache to GA Episode 106, with John Belamaric Update kernel and Kubelet config on GKE nodes AKS brings 1.17 to GA; adds containerd and priority placement group support Diamanti Spektra 3.0 Kubernetes WG Naming Introducing Cloud Native Community Groups Updated CNCF Storage whitepaper Presslabs moves to Kubernetes Presslabs Stack and WordPress Operator Links from the interview Omega Episode 43, with Brian Grant Defining scalability Original SLOs API-responsiveness: 99% of all our API calls return in less than 1 second Pod startup time: 99% of pods (with pre-pulled images) start within 5 seconds Target SLO doc - 25 nodes Borg - ~10,000 nodes Sep 2015, Kubernetes 1.0 - 100 nodes “Kubernetes Has A Ways To Go To Scale Like Google, Mesos” by Timothy Prickett Morgan March 2016, Kubernetes 1.2 - 1,000 nodes July 2016, Kubernetes 1.3 - 2,000 nodes Work by Clayton Coleman, guest of Episode 85 March 2017, Kubernetes 1.6 - 5000 nodes etcd v3 improvements for web scale Scalability Envelope Today’s scalability numbers EndpointSlices Episode 104, with Bowei Du JD.com’s 10,000 node clusters Alibaba’s 10,000 node clusters Episode 95, with Xiang Li Google’s 15,000 node GKE clusters Twitter session at the upcoming Google Cloud Next by Reza Motamedi and Maciek Różacki Poseidon and Firmament Wojciech Tyczynski: GitHub LinkedIn

Over the past 20 years, Mirantis has grown from an outsourcing company for semiconductor engineers to a product company that is the new home of Docker Enterprise. Past and present CEO and “co-founder” Adrian Ionel oversaw Mirantis’s adoption of OpenStack and purchase of Docker’s enterprise business, and he joins the show to discuss them both. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Hello Kitty, not a cat The Toys That Made Us Istanbul Not Constantinople News of the week New CNCF projects: Announcement The Future of Sandbox Sandbox project list KUDO Episode 78, with Gerred Dillon Crossplane CNI-Genie Keptn Cloud Custodian Dex Litmus Episode 56, with Evan Powell ArtifactHub Kuma Parsec BFE jFrog ChartCenter KubeCon “EU” schedule Gloo 1.4 Episode 55 with Idit Levine Frigate by Jacob Tomlinson Checkov by Bridgecrew Contour 1.6 ACI and Docker integration now public gRPC-Web for .NET now GA Episode 94, with Richard Belleville HP Ezmeral Codefresh raises $27m Links from the interview Mirantis OpenStack At Mirantis Built by NASA and Rackspace Fuel from Mirantis Adrian leaves Mirantis in 2015 Dorsal Did anyone call John Sculley? Adrian returns in 2018 Infrastructure as Code Mirantis Bring-your-own Kubernetes and Kubernetes as a Service Mirantis acquires Docker Enterprise ..and pledges to keep Docker Swarm alive Docker Enterprise Kontena closes and the team joins Mirantis Mirantis joins Airship project First release of Docker Enterprise from the merged team The Mirantis Bear Adrian Ionel on Twitter

Last week Loodse, the makers of the Kubermatic Kubernetes Platform, made that platform open source, and rebranded their company to match. Co-founder Sebastian Scheele joins us to explain how the company and platform came about, why they’ve made their changes, and what exactly a Loodse was anyway. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Docker for the new Arm Macs Tick Tock Keep Talking and Nobody Explodes Spaceteam News of the week Kubermatic 2.14 now Open Source HashiCorp Cloud Platform and new versions of Nomad, Terraform and Consul Flagger 1.0 OpenMatch 1.0 Harbor graduates at the CNCF SPIFFE and SPIRE move to incubation level CNCF post GKE goes to 15,000 nodes with Bayer Crop Science Tsunami: extensible network scanning from Google AWS App Mesh controller for Kubernetes is GA Dell announces PowerScale storage Gocker: a mini Docker written in Go by Shuveb Hussain The Kubernetes Goat by Madhu Akula Storpool and Sardina launching Kubernetes-as-a-Service Kubernetes website adopts Docsy Getting started with Oracle 18c on Kubernetes by Ron Ekins Links from the interview Kubermatic (f.k.a. Loodse) SAP HANA Julian Hansert Hamburg and Munich Kubernetes meetups ContainerDays Kubermatic Kubernetes Platform SAP Gardener Leibnitz KubeOne Loodse rebrands to Kubermatic Kubermatic Kubernetes Platform on GitHub Sebastian Scheele on Twitter

Two years ago, Sarah Wells from the Financial Times gave a KubeCon EU keynote about how the company moved from monolith to microservices, and how her Content and Metadata platform team moved to Kubernetes specifically. She joins hosts Adam and Craig to recap that migration, and what life has been like since. As Sarah has moved to a broader role in charge of all observability for The FT, she also invited Dimitar Terziev, the current platform lead for the CM team, to the conversation. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Duck eggs Green onions News of the week kube2hadoop from LinkedIn Kubera from Mayadata Episode 56, with Evan Powell Linkerd 2.8 Multi-cluster with Ambassador Consul 1.8 Intro to Istio Ingress from Banzai Cloud Cloudflow 2.0.0 Not the shoe Google internships go virtual to help Open Source Introducing the CNCF Technology Radar CNCF SIG Observability Episode 37, with Richard Hartmann Loft (and Reddit thread) Jib 2.4 announcement and Jib extensions Zerto for Kubernetes AKS 2020-06-08 adds node image upgrade and application gateway ingress controller Cloudera Data Platform for Private Clouds Cloudbees introduces DoD compliant CI, now with a CtF to deploy into an environment with an ATO, which meets DISA STIG and NIST RMF security guidelines Episode 44, with Tracy Miranda Microsoft discovers cryptojacking in Kubeflow clusters on Azure Gokul Chandra writes up Anthos Links from the interview Financial Times The pink pages Subscriber stats Coronavirus coverage The latest figures John Burn-Murdoch Added 50,000 subscribers since COVID-19 FT Crossword KubeCon EU 2018 keynote: “Switching Horses Midstream: The Challenges of Migrating 150+ Microservices to Kubernetes” by Sarah Wells Schedule Video Slides Monzo microservices graph CoreOS Fleet Innovation tokens: Choose Boring Technology by Dan McKinley Dashing from Shopify Sarah and Dimitar on Twitter

After 5 years at the helm of the CNCF, executive director Dan Kohn is stepping down to launch a new Public Health initiative. The new General Manager of the CNCF is Priyanka Sharma, who joins our show today. Priyanka tells Craig and Adam what to expect, talks about virtual events, and gives some hints on how to rename projects. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Frog Leap Studios Tubthumping (originally by Chumbawamba) Hello (originally by Adele) News of the week Rancher Longhorn is GA Fairwinds Polaris is GA AKS does new networking things Kubecost’s cluster-turndown saves you money Solo Developer Portal for Istio CVE-2020-10749: IPv4 only clusters susceptible to MitM attacks via IPv6 rogue router advertisements CVE-2020-8555: Half-Blind SSRF in kube-controller-manager Write-up from “Reeverzax” and “Hach” Ambassador 1.5 released Microk8s for Windows and Mac Finding your GKE logs by Rami Shalom and Charles Baer Business continuity with Anthos CNCF Cloud Engineer Bootcamp CKA program changes Lessons learned by Noah Kantrowitz of Ridecell Links from the interview Lightstep Ben Sigelman Ben Cronin “Spoons” Dapper Monarch OpenTracing Episode 97, with Yuri Shkuro GitLab Sid Sijbrandij CNCF Charter Governing Board members Priyanka joins as GM Dan Kohn Chris Aniszczyk On 4 years at the Linux Foundation Jim Zemlin End User Community Cheryl Hung Episode 35, with Dan Kohn LF Public Health Events: Cloud Native Summit Online KubeCon EU KubeCon Boston CNCF Technical Oversight Committee Charter Members CNCF Projects Other projects: Ollie Priyanka Sharma on Twitter

In a world where pods (and IP addresses) come and go, DNS is a critical component. John Belamaric is a Senior SWE at Google, a co-chair of Kubernetes SIG Architecture, a Core Maintainer of the CoreDNS project and author of the O’Reilly Media book Learning CoreDNS: Configuring DNS for Cloud Native Environments. He joins Craig and Adam to discuss CoreDNS, the evolution of DNS in Kubernetes, and how name resolution has been made more reliable in recent releases. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Death of George Floyd SpaceX Crew Demo 2 launch Sunniest Spring on record in the UK A small test rocket launch in Scotland UK spaceport (proposed) New Zealand spaceport (active) News of the week Priyanka Sharma replaces Dan Kohn at the CNCF Episode 35, with Dan Kohn Starboard, by Aqua Security Episode 19, with Liz Rice Docker Enterprise 3.1 from Mirantis Docker and Microsoft; Microsoft and Docker Velero v1.4 Agones v1.6 Episode 26, with Mark Mandel and Cyril Tovena Chef adds Windows container migration for GKE Red Hat adds Quarkus to Red Hat Runtimes AWS encrypts Fargate ephemeral disks in v1.4 PlanetScale open sources a Vitess operator Episode 81, with Jiten Vaidya and Sugu Sougoumarane Kubernetes provider for Hashicorp Terraform Google Vulnerability Reporting Program adds GKE Tools for debugging apps on Google Kubernetes Engine by Charles Baer and Xiang Shen How Migrate for Anthos helps modernize Java apps Helm project journey report Episode 102, with Matt Butcher Helm 3: the Good, the Bad and the Ugly by Sandor Guba of BanzaiCloud NIST deployment guidelines for proxy-based Service Mesh by Ramaswamy Chandramouli of NIST and Zack Butcher of Tetrate The World of kubectl Plugins: a YouTube series by Ahmet Alp Balkan Episode 66, with Ahmet Alp Balkan and Luk Burchard Links from the interview Domain Name System Root zone Authoritative name server Recursive and caching name server Infoblox Kubernetes Service DNS for Serivices and Pods Customizing DNS for Kubernetes CoreDNS; the default DNS server for Kubernetes since 1.11 Introduction slides KEP for CoreDNS in Kubernetes SkyDNS Miek Gieben; author of CoreDNS and SkyDNS version 2 Caddy: the HTTP server upon which CoreDNS is based Dnsmasq CoreDNS plugins Rewriting DNS with CoreDNS redisc plugin: enables a networked cache using Redis ens plugin: serve DNS records from Ethereum Name Service Node Local DNS cache and KEP BIND Unbound DNS resolver Explanatory blog posts: Understanding ndots in Kubernetes Racy conntrack and DNS lookup timeouts Learning CoreDNS: Configuring DNS for Cloud Native Environments by John Belamaric and Cricket Liu Cricket Liu and his books Book cover: a Comber fish Policy integration Episode 101, with Tim Hinrichs and Torin Sandall CoreDNS policy plugin coredns-opa SIG Architecture Production Readiness Review and KEP A DNS haiku John Belamaric on Twitter

Over the last 10 years, Cloud Foundry has grown from “open Heroku clone” to “software used at your bank”. The Cloud Foundry Foundation and the CNCF launched within a few months of each other in 2015, and the two worlds are now colliding as Cloud Foundry replatforms on top of Kubernetes. Our guest this week is the Executive Director of the Cloud Foundry Foundation, Chip Childers. He talks to Adam and Craig about foundations, the boredom of infrastructure, and the cost of every line of code you write. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Memorial Day Spring Bank Holiday Sundar Day Cracking the Cryptic: Sudoku solving and more 4 million views Craig’s favourite: watch Simon’s excitement Guardian article cheat3: Lego puzzle boxes News of the week Istio 1.6 released Multiple control planes WorkloadEntry Azure Arc for Kubernetes now in preview New AKS features GKE introduces Container Threat Detection in Beta TriggerMesh makes EveryBridge available to EveryOne in Preview Introducing KES from MinIO Updates to StackRox Kubernetes security platform OPA survey results Styra DAS adds microservices authorization Episode 101, with Tim Hinrichs and Torin Sandall Rancher Academy Understanding Anthos on Bare Metal from Google Cloud Snyk partners with Docker and Docker partners with Snyk Kubernetes Apply vs. Replace vs. Patch by David Dooling from Atomist Links from the interview DMTF and DTMF 17 year old kids asked to use a rotary phone Apache CloudStack Wikipedia, with history Apache Software Foundation Officers and Project VPs Cloud Foundry Announcement of formation GitHub Wikipedia Boeing B-29 plane Pivotal Software Linux Foundation Collaborative Projects Open Container Initiative April 2020: Chip Childers, CFF CTO, becomes Executive Director Episode 98, with Sam Ramji (the founding CEO/Executive Director of the CFF) Project Eirini: announced by IBM in April 2019 Old architecture: Diego and Garden KubeCF Created at SUSE GitHub cf-for-k8s GitHub Chip Childers on Twitter