In this episode, we cover:
- 00:00:00 - Introduction
- 00:03:20 - VMWare Tanzu
- 00:07:50 - Gustavo’s Career in Security
- 00:12:00 - Early Days in Chaos Engineering
- 00:16:30 - Catzilla
- 00:19:45 - Expanding on SRE
- 00:26:40 - Learning from Customer Trends
- 00:29:30 - Chaos Engineering at VMWare
- 00:36:00 - Outro
Links:
- Tanzu VMware: https://tanzu.vmware.com
- GitHub for SREDocs: https://github.com/google/sredocs
- E-book on how to start your incident lifecycle program: https://tanzu.vmware.com/content/ebooks/establishing-an-sre-based-incident-lifecycle-program
- Twitter: https://twitter.com/stratus
Transcript
Jason: Welcome to Break Things on Purpose, a podcast about chaos engineering and building reliable systems. In this episode, Gustavo Franco, a senior engineering manager at VMware joins us to talk about building reliability as a product feature, and the journey of chaos engineering from its place in the early days of Google’s disaster recovery practices to the modern SRE movement. Thanks, everyone, for joining us for another episode. Today with us we have Gustavo Franco, who’s a senior engineering manager at VMware. Gustavo, why don’t you say hi, and tell us about yourself.
Gustavo: Thank you very much for having me. Gustavo Franco; as you were just mentioning, I’m a senior engineering manager now at VMware. So, recently co-founded the VMware Tanzu Reliability Engineering Organization with Megan Bigelow. It’s been only a year, actually. And we’ve been doing quite a bit more than SRE; we can talk about like—we’re kind of branching out beyond SRE, as well.
Jason: Yeah, that sounds interesting. For folks who don’t know, I feel like I’ve seen VMware Tanzu around everywhere. It just suddenly went from nothing into this huge thing of, like, every single Kubernetes-related event, I feel like there’s someone from VMware Tanzu on it. So, maybe as some background, give us some information; what is VMware Tanzu?
Gustavo: Kubernetes is sort of the engine, and we have a Kubernetes distribution called Tanzu Kubernetes Grid. So, one of my teams actually works on Tanzu Kubernetes Grid. So, what is VMware Tanzu? What this really is, is what we call a modern application platform, really an end-to-end solution. So, customers expect to buy not just Kubernetes, but everything around, everything that comes with giving the developers a platform to write code, to write applications, to write workloads.
So, it’s basically the developer at a retail company or a finance company, they don’t want to run Kubernetes clusters; they would like the ability to, maybe, but they don’t necessarily think in terms of Kubernetes clusters. They want to think about workloads, applications. So, VMWare Tanzu is end-to-end solution that the engine in there is Kubernetes.
Jason: That definitely describes at least my perspective on Kubernetes is, I love running Kubernetes clusters, but at the end of the day, I don’t want to have to evaluate every single CNCF project and all of the other tools that are required in order to actually maintain and operate a Kubernetes cluster.
Gustavo: I was just going to say, and we acquired Pivotal a couple of years ago, so that brought a ton of open-source projects, such as the Spring Framework. So, for Java developers, I think it’s really cool, too, just being able to worry about development and the Java layer and a little bit of reliability, chaos engineering perspective. So, kind of really gives me full tooling, the ability common libraries. It’s so important for reliable engineering and chaos engineering as well, to give people this common surface that we can actually use to inject faults, potentially, or even just define standards.
Jason: Excellent point of having that common framework in order to do these reliability practices. So, you’ve explained what VMware Tanzu is. Tell me a bit more about how that fits in with VMware Tanzu?
Gustavo: Yeah, so one thing that happened the past few years, the SRE organization grew beyond SRE. We’re doing quite a bit of horizontal work, so SRE being one of them. So, just an example, I got to charter a compliance engineering team and one team that we call ‘Customer Zero.’ I would call them partially the representatives of growth, and then quote-unquote, “Customer problems, customer pain”, and things that we have to resolve across multiple teams. So, SRE is one function that clearly you can think of.
You cannot just think of SRE on a product basis, but you think of SRE across multiple products because we’re building a platform with multiple pieces. So, it’s kind of like putting the building blocks together for this platform. So then, of course, we’re going to have to have a team of specialists, but we need an organization of generalists, so that’s where SRE and this broader organization comes in.
Jason: Interesting. So, it’s not just we’re running a platform, we need our own SREs, but it sounds like it’s more of a group that starts to think more about the product itself and maybe even works with customers to help their reliability needs?
Gustavo: Yeah, a hundred percent. We do have SRE teams that invest the majority of their time running SaaS, so running Software as a Service. So, one of them is the Tanzu Mission Control. It’s purely SaaS, and what teams see Tanzu Mission Control does is allow the customers to run Kubernetes anywhere. So, if people have Kubernetes on-prem or they have Kubernetes on multiple public clouds, they can use TMC to be that common management surface, both API and web UI, across Kubernetes, really anywhere they have Kubernetes. So, that’s SaaS.
But for TKG SRE, that’s a different problem. We don’t have currently a TKG SaaS offering, so customers are running TKG on-prem or on public cloud themselves. So, what does the TKG SRE team do? So, that’s one team that actually [unintelligible 00:05:15] to me, and they are working directly improving the reliability of the product. So, we build reliability as a feature of the product.
So, we build a reliability scanner, which is a [unintelligible 00:05:28] plugin. It’s open-source. I can give you more examples, but that’s the gist of it, of the idea that you would hire security engineers to improve the security of a product that you sell to customers to run themselves. Why wouldn’t you hire SREs to do the same to improve the reliability of the product that customers are running themselves? So, kind of, SRE beyond SaaS, basically.
Jason: I love that idea because I feel like a lot of times in organizations that I talk with, SRE really has just been a renamed ops team. And so it’s purely internal; it’s purely thinking about we get software shipped to us from developers and it’s our responsibility to just make that run reliably. And this sounds like it is that complete embrace of the DevOps model of breaking down silos and starting to move reliability, thinking of it from a developer perspective, a product perspective.
Gustavo: Yeah. A lot of my work is spent on making analogies with security, basically. One example, several of the SREs in my org, yeah, they do spend time doing PRs with product developers, but al...