Kubernetes

We eventually realized that we needed a cloud native scheduler. And things that are not cloud native are extremely difficult to scale at the scale we want. So then you ended up basically having to build your own scheduler. The main problem that we had was basically that even Kubernetes is too small or technically GKE is too small for us. We ended up in a situation where a single cluster can scale to 15,000 nodes but we need about 12 times that. It's actually quite an interesting shift of going from supercomputers and the user workflow,. which you obviously don't want to disturb to the cloud, which kind of does things in a different way.