Geeking Out with Adriana Villela

About our guest:Edith is a Tech Evangelist at Percona, a company known for its work with open source databases. She used to work as a DevOps engineer, helping IT companies and startups set up and use DevOps. After taking a break for two years, Edith started working with Open Source, which helped her get back into the job market. She has made valuable contributions to the Apache Airflow project during her time with Outreachy and is working on translating the Kubernetes website into Spanish. Edith is also an ambassador for the Cloud Native Computing Foundation, focusing on creating content, and is recognized as a Docker captain. She has taken part in tech programs like Stanford's Code in Place and studied at 42, a coding school in California. Recently, Edith moved to the United Kingdom on a Global Talent Visa, which was a big step forward in her life.Find our guest on:X (Twitter)LinkedInYouTubeFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:Cloud Native Computing Foundation (CNCF)OutreachyApache AirflowKubernetes Community Days (KCD)Liz RiceKCD Peru - July 20th, 2024KubeHuddle Toronto 2024Additional Links:Docker Captains programCode in Place (Stanford University)42 Silicon Valley (coding school)Transcript:ADRIANA: Hey, y'all, welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. Geeking out with me today. I have Edith Puklia. And where are you calling from today?EDITH: Yeah, I am calling from UK. London, UK.ADRIANA: Awesome. I've had a few people on the show that have called in from London. I think you're like the third person from London. I had Abby Bangser, who I think it was Abby who introduced. Right? Abby is the ultimate connector of people. So thank you, Abby, for introducing us.Yes, I had Abby and then Jennifer Riggins, who is a tech journalist. You probably saw a bunch of her pieces on The New Stack. And then you. So you are my three London, UK people. Very exciting.EDITH: Thank you.ADRIANA: And we share a South American connection as well, right?EDITH: Yes. You are from Brazil, right? Peru here.ADRIANA: Yay. Home of the llamas.EDITH: We love llamas. We love them.ADRIANA: Oh, yeah, you have the awesome mug. Yeah. I was telling you earlier before we started recording that llamas and capybaras are like my two favorite animals in the world, so I always get excited when I see either one of them. Cool. Well, let's start with the lightning round questions. Are you ready?EDITH: Yes.ADRIANA: Okay. Are you left-handed or right-handed?EDITH: Right.ADRIANA: Okay. Do you prefer iPhone or Android?EDITH: iPhone.ADRIANA: Okay. Do you prefer to use Mac, Linux or Windows?EDITH: Linux. I love Linux.ADRIANA: All right. Hardcore. I love it. What is your favorite programming language?EDITH: Okay, there are many. Now my favorite right now I can say that it's Rust.ADRIANA: Very cool, very cool. I hear that it's great. But also very complicated to get into.EDITH: Yes. I mean, I don't code like a deep programming. I am just starting, just learning, but I was fascinated for what you can do with it.ADRIANA: Cool. I'm curious as a sidebar, what got you interested in learning Rust?EDITH: Because how you can easily integrate with other technologies. For example, with Docker I was trying to play, I was able to do fast with Rust. And using Chat, GPT is also a great tool to learn,.ADRIANA: Yeah, yeah, that's awesome. Very cool. Okay, next question. Do you prefer Dev or Ops?EDITH: Hard question here. Yeah, I prefer Ops.ADRIANA: Okay, cool. Next one. Do you like JSON or YAML better?EDITH: YAML. I feel that I can read it.ADRIANA: Yes. Yeah, that's my thing with YAML too. I think it's easier to read. Okay, next one may be controversial spaces or tabs? Which one do you like better?EDITH: Spaces or tabs? I use spaces.ADRIANA: All right.EDITH: Yeah. You?ADRIANA: Okay. So I used to be a big fan of tabs, but then I started using spaces, especially when working with YAML, because it felt a little bit more organic for me. Yeah. So I used to be very adamant, like, no, it's got to be tabs. But now I'm like, I'm open right now. I'm down for spaces. So, yeah.EDITH: Okay.ADRIANA: Also, kudos to you for turning the question back on me.EDITH: But I am curious about you too. Why you too?ADRIANA: I love it. Very awesome. Okay, two more questions. Do you prefer to consume content through video or text?EDITH: Okay. I love videos. I have a hard moment reading a lot of text, but videos is more easy for me to consume for you. I can imagine that too, because you do videos a lot also, right?ADRIANA: No, it's mostly text for me. It's funny, though. I was talking to my dad yesterday, so my dad does not...he was like, I do not like podcasts. I'm like, but my podcast is on video, too. He's like, it's just boring to see people's heads on video, but he's more of a video guy because he likes the visual stuff. He refuses to do podcasts. And my daughter loves, loves, loves videos. She's always learning things on Instagram or YouTube.EDITH: And you have a lot of articles.ADRIANA: Definitely. Like, I prefer writing. I think I've embraced video a little bit more. I used to be very scared of editing video, and I feel like nowadays the tools have made it easier to do video edits so that it looks like I'm not fumbling around. So I feel a lot more comfortable doing video editing compared to, like, ten years ago when it felt impossibly hard.EDITH: With writing. I feel really hard writing. Long time ago, I was not able to write a single article that take me too long to write. But now I feel I'm more comfortable because I am trying to do constantly.ADRIANA: Oh, that's awesome.EDITH: Yay.ADRIANA: I love to hear stuff like that. Final question. What is your superpower?EDITH: Patience.ADRIANA: Patience. I love it.EDITH: Yeah. You?ADRIANA: Oh, jeez. My superpower. I think I'm really good at connecting people together. I find myself in situations where I'll have a conversation with someone and then they'll ask me a question. I'm like, I know a person that you can talk to. Yeah.EDITH: You have a lot of people in your mind.ADRIANA: Yeah, I guess so. I guess so. At least remember people who should be talking to each other.EDITH: That's a superpower.ADRIANA: All right, cool. Well, that was it for the lightning round questions. You survived! Yay.EDITH: Thank you.ADRIANA: Okay, so now for the fun stuff. As I mentioned before, we got connected through Abby, and then it turns out we have another connection in common, which is we're both CNCF Ambassadors from the spring 2023 group. So, very exciting. I guess our first year of ambassadorship is coming to a close, and I guess they're renewing applications end of this month. So my question to you is, how has it been this last almost year as a CNCF Ambassador?EDITH: Almost a year because we started at March. I think the last year. I was here in London, too. Then I go back to Peru. And how I feel this year being CNCF Ambassador, I think it doesn't cost to me too much make things for being Ambassador because I was in the category. If you see there are several categories, right? Run events or you go many, you can choose whatever you want. I choose the part of content creations which I love. So when I inspire it, I just create a video. I just make a flyer or a pdf of anything which I do in my free time. And I love it because editing videos and making that things require a lot of patience.ADRIANA: Yes. There's your superpower.EDITH: That's my superpower. And I can do that. I feel really excited. I feel like I'm going to apply again. For the last month, I was not just involving in content creation, I was also involving in organizing events. We are organizing Kubernetes Community Days. Lima, Peru is the first time we are running these events in Peru with other members of the community and also being members of CFP proposals reviewers, for example. I was involved in many other things.No just content creation. A lot of things to learn. A lot of things that I never did in the past, but I never thought to do it. But I am doing. Wow, this is amazing. It's hard sometimes because it costs to learn, but it's very interesting and I like.ADRIANA: Yeah, yeah, I totally agree. And I have to say, I really enjoyed being a CNCF Ambassador because of the different opportunities that it's opened up, like just making new connections and being given opportunities to review CFPs and being given speaking opportunities that you necessarily wouldn't have had otherwise.EDITH: Yeah. I feel in the same way, just to tell you that the first trip that I did in my life outside Peru was for CNCF because I won a scholarship. So I didn't speak English, just my name. And I got to Seattle and saw a different experience. Just being in the KubeCon in Seattle, it was just amazing. And things that made me think, wow, there is doors here that I should start open. It's here I should go. I saw a lot of opportunities, and since then I go to that side of CNCF and all those communities my career start to doing that. I think the support for women in tech is also very valuable what we are doing as a community.ADRIANA: Yeah, yeah, absolutely. I do want to go back to your earlier comment on your first trip out of Peru, and you said you didn't know English at the time. How long ago was that?EDITH: I'm sorry? It was 2018. Yes. I mean, I study English. Yes, I talk basic English, but outside you is different.ADRIANA: It's different, yeah. It's so true. Because it's the slang, it's the technical terms. It's funny, because I was thinking back...as you mentioned, I'm from Brazil, but I grew up most of...I've been in Canada since I was ten. I've been in Canada for, like, almost 35 years. So I am bilingual. I'm even trilingual.I speak French, too, but I have to say my Portuguese has degraded in the time that I've been here, even though I speak to my parents in Portuguese, but I lack some of the technical terminology and I even lack some of the slang. So I actually started joining...following people on Instagram for Portuguese language school so that I can up my game to just get back into some of the slang terms and just be a little bit more conversational than I am, because I've lost some of that from not being around that many Portuguese speakers.EDITH: Yeah, I understand that. I have been here speaking English not too long time, but I already start to forgetting how to write things in Spanish, and I brought it wrong. And my father is always correcting me asterisk.ADRIANA: I know my dad's always correcting me as well, because sometimes I'll do a translation...what seems to be a direct translation of the English word to Portuguese, and he's like, yeah, that's not the same word. It means something totally different. I'm like, oh, my God, I feel so embarrassed.EDITH: You are not alone.ADRIANA: But then I remember something that I've read, like, being able to speak more than one language and making the effort to converse in more than one language is putting yourself out there. It's a sign of bravery, because, holy crap, it is so scary to attempt to communicate it in a language that you're not necessarily familiar with or super comfortable speaking in. Before we met today to record this, I recorded a podcast episode in Portuguese, and it was my second time recording a podcast episode in Portuguese. And I was so scared because I'm like, I don't know technical terminology in Portuguese. And so some of the advice that I got from a few of my Brazilian friends who live here in Toronto, they're like, "Don't worry if you don't know the word. Just use the English word, but give it a bit of a Portuguese accent." Yeah. I mean, like, you know, even though, like, something like that completely scared the shit out of me. At the same time, I'm like, you know what? I'm going to force myself to do this because the more I do it, the more comfortable I will get.EDITH: Yes. I don't know why we are like that. I mean, we are really afraid. We jump and we start to doing. Then it pass and we said we did it. Yeah. Before that start to feel like the fear, the hands start to with everything, that scary moment. Then you use go, but then you jump to another thing.EDITH: To start to jump to a ring and another ring. The same motions.ADRIANA: Exactly. It actually reminds me of, like, I was having this conversation last week with someone where I'm like, oh, my God. When I first learned about cloud and cloud native, I'm like, it's this terrifying, scary thing. So I was like, I don't want to do it. I don't know. I don't think I can do it. And then I did my first thing in the cloud and I'm like, oh, okay. It was okay.ADRIANA: Yeah, it wasn't scary.EDITH: You are complete. Nothing happened. It's weird how we can be afraid of things that also involve human beings, like communications, like speaking, we are afraid. I don't know what we are afraid. What is the fear that we feel to be exposed, to see that others look at us and we are trying to embarrass. I think we all are humans and we all have the mistakes.ADRIANA: Yeah. And I think we judge ourselves a lot more than others judge us. When I'm having a conversation with someone in Portuguese, especially like, with my family in Brazil, and thankfully know Google translate to help me when I'm on WhatsApp, but I'm like, oh, my God, they're going to look at me and they're going to make fun of my grammar, whatever, or use the wrong word. But then I also have to remind myself they have better things to do than to nit-pick on your grammar. They have their own lives. Get over yourself. It's not all about you.EDITH: Yeah.ADRIANA: Okay, so I want to switch gears again and talk a little bit about your career, like how you got into...and I know you do a lot of work around Kubernetes and containers. What got you in it?EDITH: Yeah. Okay. I was in the field of tech for almost ten years. I can say I work it as a DevOps, also as a developer for big companies in Peru. For companies where I started from scratch, things. Was really hard. For example, when DevOps was not big tendency. Right now we are starting from scratch. I started from scratch alone.Trying to start servers, make all that stuff was really hard, but challenge. And after that I decided to quit my job in 2018, I think...2019. Because of healthy problems, emotional problems, healthy problems, back problems, and with family problems, everything like when you have one and everything start to make a big thing. And I decided to take a moment. I take two years. I never thought it will take me too much, but I took two years. Okay. But these two years was really amazing for me. It was amazing because I give me this time to know me better.Things that I never did in the past. Because I was always running, running, piecing the car. I don't know how to say the accelerator of the car and trying to gas in that life. But then when everything happened, I just. No plan. Nothing for that future, for the future, just that. Just myself, my thoughts and my body. And thinking what made me happy, what will make me happy for the future.It's how I invest the time in two years. So not just thinking, but also doing. Because I wanted to improve English, I wanted to improve also my technical skills. And I realized that tech made me happy. It's one of the things also make me happy. Okay. I'm also geek.ADRIANA: I love it.EDITH: Yeah. Between several things, tech also made me happy. And I start to improve my skills. I start to learning English, which was really challenged for me. Now I can communicate how I want. I think I need to improve, but it's good for me. So I started to apply for jobs after having an internship in Outreachy. Did you hear about Outreachy?ADRIANA: Yes, yes. I have heard of Outreachy. For folks who have not heard of Outreachy...EDITH: Yes. Outreachy is a program, open source program. In three months you can have a mentor. It is also paid. So you learn a lot of things because you put your hands in real open source projects. I put my hands in Apache Airflow, where I start to code. I start to make things that I had never thought to do it. It was really amazing. And I wasn't with Oyo, but they give me a pay.So it was enough to me to survive and to learn English and improve some soft skills and also technical skills. Then I started to apply a job. I set a goal for me, for myself, to apply for an international company where I can speak English. Have that opportunity to speak English. So applied maybe to 200 jobs in two months. I applied the most I can.ADRIANA: Wow.EDITH: Sweden, Germany, USA. I send my CVs a lot. So one of the companies was Percona, and after the process and everything, I was hired by Percona and now I'm working as a technology evangelist in Percona, which is an open source company.ADRIANA: That's so cool. And I have to say, it so resonated with me when you said that as part of your time of really digging into who you are and what you love, that you decided that you love tech. Because I felt like I went through a similar thing in my career as well. I was working at a bank and I had quit my job at the bank to become a professional full time photographer. And I was like, this is it. I'm done. I don't want to work in tech. I want to do photography.This is my passion. And I did it for a year, and then I came to this moment in my life where I was like, so it's really hard. And if I really want to make this work, I can probably give it another year or two and probably finally start seeing growth. Because at the time, it was like I wasn't really right then I thought, but do I want to invest this extra time to grow my photography business? What do I actually like doing? Then I realized I had more fun tinkering around, like doing my newsletters and tinkering around with my website, and I was using WordPress and I bought this plugin that wasn't working. And I'm like, let's go into the PHP code to see what's wrong. And I'm like, oh, I think I like that more. So I ended up like. I'm like, you know what? I want to go back to tech.It took a year of me not being in tech to realize that I actually enjoyed tech. So, anyway, yeah, your story so much resonated with me, and I think it's so awesome and so important to take the time in our careers to figure out what makes us happy because, I don't know, we're at work for most of our lives and it better be something that we enjoy, right?EDITH: Yeah. And it's different and it's unique history. I can say yours, for example, is totally different than mine, but it's very unique. It has the meaning for you, and that is the good thing, the very important thing that maybe nobody's going to understand. But we are going to understand, right?ADRIANA: Yeah.EDITH: We are the only one who understand that special moment. Yes.ADRIANA: It's so true. Very deep thoughts. I love it. So philosophical. So great. The other thing I wanted to ask you about, because how did you get into doing Kubernetes work?EDITH: Kubernetes in Peru, we started to hear about Docker, for example, we started to see the whale everywhere. What is the doll? And that was the curiosity, the doll with a terminal. The terminal to Docker and containers and all that stuff. The same, I think happened with Kubernetes. In Peru, we started to listen about Kubernetes as a technology, as a standard things, but just listening. So I started to follow. What is this Kubernetes thing that people talk? And I start to follow people on social media, like Liz Rice, for example. The first person, people that I was following was Kelsey [Hightower].You interview Kelsey. Kelsey, Liz. That's big people there. So I was really fascinated for the keynote that they did. So I started to investigate about KubeCon, and it's how I got the scholarship to go. And once I go there, I see, wow, Kubernetes is the thing. So I started doing some demos at home with Google Cloud because I had free credits. So I start to play because it's what I want to. I love to do, to play with technology.Do it, destroy it, create it, destroy it. Mac, Linux, Windows, destroy it again. I don't know. But I found it funny. Funny, yes. Funny. Enjoyable? What is the word? Okay.ADRIANA: Yeah. Fun.EDITH: Yes.ADRIANA: That's awesome. It's so cool. And I especially love what you said about creating and destroying. And I think that's honestly some of the most fun stuff about playing with Kubernetes clusters is like, you do a bunch of stuff, you mess it up. Okay, time to start over again.EDITH: We are not in production, so you can destroy.ADRIANA: Right, exactly. Yeah, definitely don't do that with your production cluster. So you mentioned playing around with Google Cloud. Have you played around with any other cloud providers?EDITH: Yes, I had the opportunity to play with Red Hat, with Amazon, with Google Cloud and Azure. Yeah.ADRIANA: Cool. And which one's your favorite of the ones you've played with?EDITH: Google is my favorite. I don't know. I feel like interface, the graphical user interface, was more, for me, easy to do it, easy to create, to understand. For me, I feel that in that time, I feel that Amazon has a lot of things. Maybe that I didn't get too much distracted. But anyway, I use in the same time the three cloud providers.ADRIANA: Cool. That's awesome. So switching gears a bit, I wanted to talk a little bit about some of your community work.EDITH: Thank you for that question and community. I started in community participating as all of us just going to the events and see people talking and watch. But then I say, okay, there is another weight because you are in a certain level I can say you advance a little bit in your career and you say, okay, there is people who did a lot for you. They give time, they prepare. So you learn. So let's make it something for that too. And it's the mindset of the community, right? Get back this kind of things. So now we started with creating communities.I say we because it's not just me, we always work with people in communities and we created communities in the city where I was living. There was where I was living lacks of community techs. There is no much communities in tech. So it's where I wanted to start. I'm going to create communities with many people. So Docker was one of the companies that helped me to make it with sponsoring some events. So we start to create events for per year. For example, we celebrate the anniversary of Docker. Like, the 10th anniversary which was a lot of people going to that event and they are learning about the technology and that is one of the work that we are working until now.I like of that and I feel proud about that because we are doing something small but maybe could be impactful and give this opportunity to people that don't have the opportunity to make in that city without leaving the city. Yeah, this is one of the things that we are doing and the other is CNCF. I love this. So we had this big opportunity also because CNCF sponsor it. We have all the support of CNCF to make it possible a Kubernetes Community Days in my country in Peru. So we as a team because we are several people working in that we are creating this community for this year, for July. So I hope we can see it and we can repeat it over the year. So this will be impact also and generate more opportunities for people in our country.ADRIANA: That's amazing. Now how much work goes into putting together a Kubernetes Community Day> But actually before I get you to answer that, maybe it would be helpful to explain to our audience what is a Kubernetes Community Day? What's the purpose of having something like that?EDITH: Yeah, these are spaces where we give people the opportunity to share about the expertise they have about the Kubernetes and the CNCF ecosystem that exists. So a Kubernetes community days is an event. Could be in person, online or both, two days or one day. We choose that. And where several experts or people who want to share about ecosystem of Kubernetes go and start to talk about that. Could be not just talk, could be workshop, could be several things lightning talks, open forum, things like that. And sometimes it's free, sometimes it requires some payment. It depends on the organization, but it's a big opportunity to join a lot of experts, beginners, enthusiasts, members of communities between all this ecosystem. Kubernetes ecosystem.ADRIANA: That's amazing. So it's basically like a little mini conference.EDITH: Yeah.ADRIANA: It mini though? It sounds like. It sounds like a lot of work.EDITH: We compare it with KubeCon, could be mini, but to be honest, it's not like to be mini. Not mini like I saw 500 people in some of the Kubernetes Community Days in Europe, I think.ADRIANA: Holy cow. Damn.EDITH: We are targeting in Peru for the Kubernetes Community Days in Peru, we are targeting also 500 people. Yeah. Attendees.ADRIANA: Amazing. That's so cool. And so for organizing Kubernetes Community Day or KCD, what type of support do you get from the CNCF? As a CNCF Ambassador I would imagine that you get a little extra boost of support from the CNCF? So if you could talk a little bit about that?EDITH: Yeah. What we have is support from members of the CNCF, people who work there. So they help us organize and we have synchronization meetings sometimes to see how is our progress. Also they try to support us the most they can. For example providing us the logos and designer people who can also help us. They also sign a budget for coupons, courses, coupons and some budget. I don't remember the amount of the budget to start the event. That will help us to pay some things and what more? I'm not sure about that but they give the opportunity also to travel to the KubeCon I think.But maybe I am wrong. I'm not sure about that but I think there is many opportunities. Once you are in the ecosystem and once you are doing things there are many opportunities. Networking is also a big opportunity because in an event you can contact with several people who also are organizing. This is my first time organizing so I don't have precise response how much that will take me because it's the first time that I am running it. Let's see how it goes.ADRIANA: So does the CNCF provide then the overall funding for running a KCD or do they provide some funding? Do you need sponsorships? How does that work?EDITH: Yeah, we need a sponsorship. Each team tried to find a sponsorship in the country or outside the country. So with that budget is how they estimate how many attendees we will have and how we are going to assign it. In some cases, this is free and the budget that you need is maybe less, right? It depends, to be honest, of the country and of the city of the country, because the governance community is now is for city. So let's give the opportunity to have more in a country.ADRIANA: Cool. That's awesome.EDITH: Did you think to organize an event? Did you think to participate?ADRIANA: So I'm actually helping to organize an event in Toronto called KubeHuddle, which is like...I think the first KubeHuddle took place in the UK, I want to say a few years ago. And then there was a KubeHuddle in Toronto last year that I attended as a speaker. So then the organizer of KubeHuddle, Marino, he asked me at the end of last one, he's like, "Do you want to help organize the 2024 one? I'm like, okay, yeah." So I am involved in that...because I have so many things on my plate, like, I'm trying to take on what I can without being overwhelmed, but still making sure that I help out. So this is my first experience with that. And KubeHuddle is taking place on May the 7th in Toronto. So this year it's going to be a one-day conference. Last year it was a two-day conference. This year it's a one day single-track conference. So yeah, very exciting. So is KCD Peru? Is it a one-day or two-day conference?EDITH: One-day conference.ADRIANA: One day. And is it multiple tracks or is it single track?EDITH: Multiple. We are thinking multiple.ADRIANA: Okay, cool. Awesome. Very exciting. I'm super stoked for you. I hope it all goes well. Now we are coming up on time, but before we finish off, do you have any parting words of wisdom for our audience?EDITH: If I can say something, it's enjoy life.ADRIANA: I love that. That is perfect.EDITH: See the sun. Look at that and enjoy it. It's very nice. Sometimes. If you have sun.ADRIANA: Except on cold days.EDITH: It's really cold. There is no sun.ADRIANA: I don't know...What's the temperature like in London today, because here it's a warm -4C.EDITH: Today there was a sun, but once you put the finger outside, it freezes. But the sun was lining.ADRIANA: That makes it better. Yesterday it was like -15C in Toronto and I went for a walk and I had to go into different stores to warm up. So I didn't freeze. But yes, I absolutely love your parting words of wisdom. I think we get so caught up in our work lives that we forget to also just take a break, reset, enjoy life. Enjoy the non-work time. Well, this was awesome. Thank you so much, Edith, for geeking out with me today. Y'all don't forget to subscribe and be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time...EDITH: Peace out, and geek out.ADRIANA: Geeking Out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who, incidentally designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.

About our guest:Jennifer Riggins is a culture side of tech storyteller, journalist, writer, and event and podcast host, helping to share the stories where culture and technology collide and to translate the impact of the tech we are building. She has been a working writer since 2003, and is currently based in London.Find our guest on:X (Twitter)LinkedInMastodonBlueskyFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:The New StackLeadDevAt QCon: Why Generative AI Is Harmful to Earth and SocietyE-lanceKelsey Hightower at Civo NavigateAbby BangserDiversity, Equity, and Inclusion (DEI)David Heinemeier Hansson (DHH)37signals (formerly Basecamp)GitHub CopilotBackstage (CNCF Project donated by Spotify)Divya Mohan (Kubernetes Maintainer)Octoverse: The state of open source and rise of AI in 2023The AI Revolution Is Just Getting Started: Leslie Miley Bids Us to Act Now against Its Bias and CO2Consequence Scanning – an agile event for responsible teamsAdditional Links:Developer Empowerment Via Platform Engineering, Self-Service Toolingtag-environmental-sustainability Slack Channel (CNCF Slack)Transcript:ADRIANA: Hey, y'all, welcome to Geeking Out, the podcast about all geeky aspects of software delivery. DevOps, Observability, reliability, and everything in between. I'm your host Adriana Villela. Coming to you from Toronto, Canada, and geeking out with me today is Jennifer Riggins. Welcome, Jennifer.JENNIFER: Hi, thank you so much for having me on.ADRIANA: I'm super excited to have you join me. And where are you calling from today?JENNIFER: London.ADRIANA: Awesome. What I'll do is we'll start with some lightning round questions and then I'll get you to talk a little bit about yourself and then we'll go from there. Sound good?JENNIFER: Great, yeah, sure.ADRIANA: All right, let's do this. Okay, first question. Are you a lefty or a righty?JENNIFER: Righty.ADRIANA: All right, do you prefer iPhone or Android?JENNIFER: iPhone. Just because it's what I have and it's seamless. It's not a moral choice, but it's a convenience choice.ADRIANA: That's fair. Next question. Do you prefer Mac, Linux or Windows?JENNIFER: Mac. Same. Convenience.ADRIANA: Convenience is always very important. Okay, next question. As a tech journalist, do you lean towards Dev or Ops?JENNIFER: Oh, Dev. Well, no, that's hard. No, I would say either side. Yeah, because Platform Engineering is all about bridging that gap, isn't it?ADRIANA: Yeah, that's very true. Exactly. Okay, next question. Do you prefer to consume content through video or text?JENNIFER: Text for sure. Or audio more than anything. Podcast.ADRIANA: Yeah, I love me a good podcast. I have like way too many in my queue that I have to get through. Okay, final question. What is your superpower?JENNIFER: Connecting people, introducing different people that can help people figure out their next step or their next job or people should just know. People. Yeah.ADRIANA: Awesome. I love that. I think it's so important. I think people really underestimate the power of connection. All right, so we are on to the main event, the meaty bits, if you will. So why don't you share with our audience what you do with TheNewStack?JENNIFER: Okay. I have been a working writer since uni. I am not a trained journalist. I went for political science and I've been in the tech niche for 12 or 13 years. That includes both the marketing side and journalism side. I'm just a naturally good writer and good at explaining complex topics so that everyone understands, which is good because I'm geek by association, I am nerdy by nature, but I am not technical. So it helps me then help other people understand because everyone should be involved in understanding the future and how it's being built, especially as it gets more pervasive in our bodies. In our homes and our cars and then AI thinking for our behalf, on our behalf, et cetera.JENNIFER: And I have been writing for various as a freelancer, but with The New Stack for over eight years now, so pretty much their first year. And also I write for LeadDev and other blogs and then have software customers, things like that, helping them do their case studies or explain. I am not interested in funding, not interested in who's appointed CEO, not interested in crypto, not interested in technology precisely. I'm much more interested in the cultural impact of technology and what it's done. So I won't typically write about a new feature unless something extraordinary is about it. But I will write about once that feature is used and how it impacts people's lives, or more feature-driven like thought leadership, things like that.ADRIANA: Cool. That's awesome. So you mentioned that in university you did not come from a journalism background. So how did you find yourself writing for a living? Like you said, it came naturally. What gave you the first opportunity?JENNIFER: I've always been a natural writer, but I'm good at writing in that side. "Soy de letras," as you would say in Spanish. Math is how you would say it in English. And I was actually editor of my school newspaper and all, at university, so I was always involved in some way in writing and in helping other people write better things like that. So it's just a natural thing for me. I've always been able to fall back on writing.ADRIANA: And then how did you find yourself, like writing about technology then?JENNIFER: What else is there to write about? I think role was through Elance, or whatever it's called now. One of those Upwork, one of those freelance websites, and from there it spiraled. Something I'm good at explaining complicated concepts.ADRIANA: I think there's not enough emphasis on really being able to distill things in a very approachable manner, right? Especially a lot of docs out there, technical docs are so.JENNIFER: Complicated and incomplete at the same time.I think it's the most important thing. Critical thinking and being able to talk across that chasm or chasm between technology and business will be the greatest skill set and is so important, especially in this time of AI, because you need to be able to distinguish the bullshit that the AI we know is giving what, 52% of code generated by Chat GPT is wrong, but Chat GPT is very convincing because it was trained by tech bros, which have great sense of confidence and to sell bullshit. So it doesn't have to tell you when it's wrong. So in this time when we're entering AI and all this productivity mentality and everything, we need to be able to understand, be suspicious of what is working or not. And we also need to understand the business impact. So either side of it, whether it's business needing to understand that wildly expensive cost center of engineering and cloud, or engineering being able to explain and feel connected to that business impact and to understand, so everyone's going to have to explain to themselves. And Kelsey Hightower said at Civo Navigate, an event...he said, we have this weird, maybe it's a corporate throwback, where in tech we're like, I have this great idea, but I'm not done my slides yet, I'm not done my PowerPoint presentation yet. We'll wait to talk about it.But that's not how things work. People are storytellers. People need to be able to have conversations, even if it's expressing yourself in writing. I don't think it's necessarily very inclusive at all that everyone has to speak on stage or speak, but one-on-one conversations is still going to be a very important thing. And being able to write, even in Slack and be concise, so that's not my strong suit because I write very long features and things like that. But being able to express yourself in a way that everyone understands, because especially with AI, as we get into this interstitial age of prompt engineering, the next maybe two years, it's going to be the subject matter experts that are really important. So you won't need necessarily for everything, a coder. But if it's like building management or security in a building, maybe you need someone that actually has experience in that, who can work and partner with the developer to build something that's actually useful in AI.ADRIANA: Yeah.JENNIFER: So they need to talk to each other. And the people that may be deciding, especially with a chat bot, customer support and all, may have zero coding capabilities. So you need to be able to talk and communicate with them. And that's where the benefit from AI will come about. And it's honestly where we're going right now.ADRIANA: Yeah, I think the interesting thing is AI, in a way, keeps us on our toes because you almost have to be smarter than the AI to be able to pick out the bullshit, right? Because the minute you start trusting the AI and what it produces, that's what gets you in trouble, right?JENNIFER: Absolutely. And it's just different. We forget Chat GPT specifically is a large research project. It's not a tool. You are part of a research project. The tool is when you pay for like a private version of any of the AI tools that are trained on your context, your documentation, your processes. That's where the value comes. So if it's free, you should probably distrust it.JENNIFER: And also think about how bad that is for the earth.ADRIANA: Yeah, absolutely. I totally agree. Now, on the same vein of Chat GPT, I've heard initiatives from various companies where they want to replace a chunk of their written content with AI-generated content. What are your thoughts around that?JENNIFER: Okay, so in the world of documentation and things, I think it's very interesting. I think that is...documentation writers are super important, but there's also a lot of companies relying on developers to create docs. And in the 12, 13 years I've been in the industry, I started out a lot in the API space. Number one complaint was that there was not enough documentation. Yes, the number one thing developers don't want to do is write documentation. So having documentation embedded next to the code and somewhat AI-generated I think is very valuable. Human-generated media, things like that. There was a rumor 95% of media will be generated by AI by 2025 and all.I think we're having a real backlash about that. I know AI can't do what I can do, and I don't use it that much. I don't really use it. But my understanding, when other people use it and all, it's for the low value content. Have a proper conversation with someone to distill from someone that maybe isn't as easily expressing themselves because maybe they've got a very technical mindset. It can't have that conversation and draw out of them the true value of their product and then translate it?ADRIANA: Yeah.JENNIFER: Could it be useful if someone wrote an article themselves and then wanted to from that article spew out a bunch of social posts or something? It could probably be very interesting for that. Just very suspicious and controlling. You have to be anyway. But when you go through all of that, I don't feel my job is going to be in trouble. The people whose jobs are going to be in trouble are people whose lives live in Excel. Things that can and should be automated. The point is that we work on real problems. Boring, low level-coding problems will be automated, like repetitions.Creative work should get more creative, more problem solving. But then the boring stuff, I don't know what I could automate. I'd love to automate. Like invoicing, because I tend to procrastinate that because again, soy des letras. I'm not good at math, but then I don't trust the systems to throw that private information in there.ADRIANA: Yeah.JENNIFER: Also, we cannot forget that there's this unbelievable inequality that's being caused by data centers. It is causing a huge environmental impact. In west London alone, affordable housing cannot be built. There can be no new affordable housing in one of the largest cities in the world, one of the alpha cities, because too much power is being taken by data centers.ADRIANA: Wow.JENNIFER: To cool them down, et cetera. They're super polluting. Like, it's really bad. Note that I said affordable housing. So rich people who are leaving these plots empty and funneling money, because London's like a huge money laundering area, those are still being built and left empty. But people that truly need homes cannot get homes in west London because, specifically data center power. So I think we need to think about how we're impacting the environment. There's very interesting things going on for FinOps and optimizing your Kubernetes clusters, not getting in this habit of being double the amount of cloud just in case, but having things.And this is where AI is very interesting too, because AI can be a solution to help. It's always better to have the tool manage it than a human manage that, because if a human is responsible, they're always going to give more, just in case. They'll never give less, but they'll always more. So that's where AI can be a solution or part of the solution. But we should be putting far more pressure on anything we're paying for. We should be putting pressure as a customer that they are putting on data centers that are sustainable.ADRIANA: Yeah, I think we have to sort of move away from this mentality, as you alluded to earlier, of just more and more and more throw more at it, because it's like infinite resources. First of all, it costs money. If that doesn't deter you, which it should, then think about the environmental impact, which is just absolutely mind blowing.JENNIFER: And then that leads to another impact that disproportionately negatively affects people from underrepresented groups. Whether it's pollution in Virginia, which has a very underprivileged community, very impoverished community in Virginia that are directly...have hearing problems, have asthma problems, these are all problems. So yeah, I think we need to consider, in everything we do as tech storytellers, we need to consider the implication beyond the stereotypical developer, but we need to help them think about who will most likely be harmed by this and who will be more likely to be excluded or what being near.ADRIANA: Yeah, I completely agree. When you're writing an article, what inspires you? How do you decide what to write about?JENNIFER: It's 50/50 now because I've been writing so much about developer productivity and Platform Engineering, and, before DEI, but no one cares in 2023 about DEI. See the numbers. Sadly, diversity, equity, and inclusion is not a priority, so you have to do it surreptitiously, like by who you interview and stuff. Can't just write directly about it. I get reached out to a lot. I also see people's talks or use LinkedIn a lot. So there's all that.ADRIANA: And then the other thing I want to ask. You said that you do a lot of writing on Platform Engineering. What got you interested in Platform Engineering in the first place?JENNIFER: Oh, it's really a simplistic thing. I've been writing about and working in the Agile and DevOps space for a really long time. I write about culture side of tech, and like I said, in 2023, I see it in the data, I see it in traffic and all. Tech isn't even trying to pretend they care about diversity, equity and inclusion anymore. But you know what? Look at it while women, and that's probably the most privileged, minority or minoritized group in tech. While women make up about between 22 and 24% of the industry, there were 69% of layoffs. Black startups are not getting funding. I mean, it went from abysmal to 0.0002 abysmal percentage.ADRIANA: Wow.JENNIFER: People like Elon Musk and DHH from BaseCamp, they've made it cool publicly to not give a fuck about diversity, equity, inclusion. That means before it was informative...sorry...that means, before it was performative, but now they're not even trying to be performative. So there's that. And there's been a ton of cuts and layoffs. I see those cuts because there's two things. There's the last hired, first fired. So if they only started caring about diversity in the last two years, well, those people are going to be first cut. They also tend to be in roles like DEI, which were cut across the board.Accessibility cut across the board. Marketing, at least perennially, is cut when there's cutbacks, but tend to be more people from minoritized groups. But on the other hand, what's 2023 been about? A lot about tech layoffs, which means a lot of trying to do more with less. And then on top of that, the code is just getting more and more complex. The cognitive load is more and more extreme. And I think while we...we, not me.But the tech industry in general, doesn't seem to care about diversity, equity, inclusion, accessibility as much anymore, sadly, it does still understand, and I don't know that we can go back to, they've tried to return to office so many times and guess what? People are not happy, they're not productive, they're going to leave. Yes, the hand is more of an employer's market, but is still an employee's market across the board. And there's all these things where companies are realizing what statistics and data and journalism has said for years, that happy workers are more productive. And that doesn't mean massages and ping pong tables or foosball tables. That means actually finding purpose in your work, having visibility, not having even logically, from a nutty corporate standpoint, not having so many distractions and all the meetings blew up. So there's all of that. So there's this push for developer productivity because budgets are tighter, people need to make more money, staff is still bigger than it was a year, maybe two years ago. There was this irresponsible, cannibalistic growth for a while there, and it's kind of a correction, but the code has grown in the meantime too.The cloud native landscape is obscenely complex. So there's this idea we need to work on developer productivity, which is where Platform Engineering comes in. Instead of being a platform that we've had for... since codes exist. Like Cisco was making platforms back in the '70s. It was, you do this, you control this, which for some security stuff is not a bad idea for role-based access control and all that should not be optional. But the majority of the idea of Platform Engineering is that your customers are your developers and you are building a platform as a product where you are getting feedback from them constantly and you're building just what they need to get better. And then also it comes back to that whole docs problem. What is a huge problem? Who is breaking that developer flow, that getting in the zone is not being able to find things, googling it, going to Stack Overflow, asking a question on Reddit. Instead you've got this...we haven't even mentioned Copilot yet, but I think that for the developer audience has the most potential, because it's in with where 85% of repos are...in GitHub. So it's about them not context-switching as much and meetings actually having value, not having Agile.And then Covid just led to this multiplication of meetings for meeting. So Abby Bangser from Syntasso has my favorite definition of what Platform Engineering is, which it's almost like a physical platform you're supporting people on that takes care of the not differential but not unimportant work. So with DevOps, we went through this idea that you build, you test, you maintain, you do all of that, all the way to the cloud, all the way to release and all. But cloud is not differential to the average programmer, specifically to their audience, which would tend to be external users or customers. Security, very important, not differential testing. Very important, not differential repetitive work. Now it just should just be automated. So it doesn't matter anyway.And it's about...Spotify calls it Golden Pathway. I like calling it the Yellow Brick Road because if your developers wander off, they may go in a poppy field and go down a Reddit rabbit hole. But if Dorothy and them had stayed on the Yellow Brick Road, they would have been a lot faster. If Gandalf had given the eagles from the start, the book would have been a lot shorter. So why don't we do that? Guess what? If you had asked what Frodo would like? Oh, that's a new nerdy euphemism I'm coming up with right now, metaphor. But I think it works. Would have been a lot shorter movie, a lot shorter movie series, book series, and probably a lot more people wouldn't have died.So just ask your developers what is frustrating them and then start there.ADRIANA: Yeah, exactly. And there are so many things that frustrate developers.JENNIFER: And [inaudible] and searchability are always at the top of that list. They want to know who does what in a company, which again, comes down to collaboration and knowing people across the business. It's a positive thing to learn.ADRIANA: Yeah, absolutely. And there's another one. I think it came about from a question that you asked on one of the socials, which was something around, what are some of the developer frustrations? And I was thinking back to so many jobs where I started off...and onboarding and setting up a new environment on your machine is like the most fucking irritating experience ever. It's like, why do we have to keep doing the same thing over and over and over again? Why don't we have a streamlined process for setting up our dev environments when we start a new job?JENNIFER: Why would. Yeah, why would you even need to, why is setting up an environment useful for you to be doing? It's not helping the customer, it's not driving value. So Spotify, being like one of know, they created Backstage and outsourced it because they thought it was that important to standardize it in the community, which I like. But by them using Backstage, they got their developer onboarding time, which I believe they count as ten pull requests. Like that is when you consider productive. They went from 110 days to 20 days, pull requests because you just get people up and running. You give them what they need. You wouldn't give them a laptop and have them install Windows or install Linux or install whatever you want on your laptop. Give them the tool.ADRIANA: Yeah.JENNIFER: So just do that for all of the cloud because, and then you still give them the option. There will still be your 5% that want to engineer their way around a problem. And that's why you build it with APIs and you let people do their own thing. But maybe you don't need to support their work. They're at their own risk. They're on that poppy field, they're doing their own thing. But you'll support that 95% and that's okay.ADRIANA: Yeah. I really love your analogy of the Yellow Brick Road, because it really is all about like, these are your guardrails. It's there to protect you from yourself. Because we like to deviate. Sometimes we're not necessarily aware that that's not a great thing to do.JENNIFER: And you can still deviate. That's why you, as a Platform Engineer have to make something they want to use. And again, it comes all the way back to that tech storytelling, those early wins, the examples. Just the proof of good work is you need to make something they want to use. And then you have your customers who happen to be internal, probably more annoying, but you have a much tighter feedback loop. So you're going to get more direct feedback all the time. It's a good thing. It can just be probably a bit awkward for some people.Also, there's the problem that Platform Engineers are engineers, so they think they know best, which is not the point. And you just build something that they want to use, make it easy for them to stay on the path. So even the guardrails, I picture that car cannot really go past those guardrails. Follow the lines.ADRIANA: Yeah, it's like this is the path with some flexibility in mind, but you only have...JENNIFER: Fall off the cliff, and that is all you.ADRIANA: I think that's a perfect analogy. I love that. And the final thing that I wanted to touch upon, and you brought it up a few times, and I think it's actually a very important subject, which is DEI, which, as you pointed out, is the conversation around it has changed a lot, but the problem still remains. And it's kind of interesting because...I've had a number of conversations with people over the years, and after you pointed it out, I'm like, yeah, I guess it's kind of unfashionable to like, oh, let's have the panels of underrepresented groups talking about being underrepresented. Then it's like, well, as you said, we have to do it in a sneaky manner. But I think we do have to call it out for what it is because you go to tech conferences and I was a speaker at Observability Day, the co-located event for KubeCon North America, and there were three of us female speakers for all of Observability Day. And I was like, what the hell?JENNIFER: Could probably guess two of them just by knowing the handful of females or women that have access to that space and who are doing amazing work. But yeah, we don't need VIP bathrooms at tech events, we need representation. It's the only time we would be very happy to queue at bathrooms. Please, tech events.But like anything in the. When we're talking about open source, 3% out of what, 20 speakers or something for co-located day, it's actually not a bad percentage for open source because open source around 4% women and non-binary because it's toxic, because it's based on free work, which we do the brunt of anyway.ADRIANA: So true.JENNIFER: Women and people of color are far more likely to be doing free voluntary work and they don't have time for it. But then you lose the benefits of public code samples, of working with companies that actually are really big companies, like a Google or a Spotify or Atlassian, all these companies that support a lot of open source or access Amazon Web Services. These are companies that provide a lot of open source. But then if you can't go to these events, you can't work on these projects because you can't do free work. Open source is a huge problem. So it's always going to be worse. Which open source should I believe that open source should be free code, but I don't think believe in free labor, and I think that's a huge problem.ADRIANA: Yeah, absolutely.JENNIFER: You are a company benefiting from an open source project. You should be investing.Either find a way to sponsor that project or hire a staffer that contributes to that project as their deal, as their job, and just also focused on both technical and nontechnical contributions. Because again, we're back to documentation, we're back to the other big barrier to entry in open source diversity is that everything's in English. So you need people translate. Another use case that in probably 18 months will be very valuable from AI.ADRIANA: Yes, we take it for granted that we're English speakers, so we're like, yeah, of course, no problem. But I do remember, I think it was someone at KubeCon who was saying that they felt so shy about contributing to stuff because English wasn't their native language and they know incredibly smart, but they just didn't feel confident contributing to open source. And it just. Oh, my God.JENNIFER: Even in other languages, you need to know English too, to be a translator because it's the de facto language to translate to. But for example, Kubernetes, which Divya Mohan runs with someone else. I forget their name, sorry, but has organized for years the documentation translation, and it's across like 18 languages, or will be soon. Zero are in Africa. Are African languages zero?ADRIANA: Oh, wow.JENNIFER: Only about 2%, maybe 3%, depending on what you see of open source contributors and users are from Africa, which is about 19% of the world population and likely the geographic area that would most benefit from free and open and secure software, because typically open source is also more secure, more eyeballs, more people involved, et cetera. So it would benefit everyone, like, at an exponential GDP level, but because it's just in English...ADRIANA: Yeah. And it occurs to me also that even our programming languages...the syntax is in English!JENNIFER: And doesn't seem like that's going to change. Yeah, no, that is where AI, I think, will be interesting.ADRIANA: Yeah, it'll be definitely very interesting to see where it goes. Now, as we wrap things up, do you have any final thoughts on where you see this industry, our tech industry, going in the next, say, year?JENNIFER: That's it. It's a year, year and a half tops, because we're in this transition period where AI is still nascent, but it will very quickly advance and it will be much more useful because it will be context-specific, and I hope it won't be companies like...Telephonica in Spain fired, like, a huge chunk of its customer support reps because it's like, we can just use a chat AI. It's not great. I'm an HSBC customer, and I'm always like, give me human, give me human.ADRIANA: Yes.JENNIFER: It's not working. The Moby whatever, the chat bot thing, they. It's. It's not for me. I know a lot of people would rather talk to a bot, definitely, than stay on hold, but it's just not there yet. So we need humans in the loop now more than ever who have that subject matter expertise. We're not there yet, but we then need real humans in the loop feeding back into the AI, whatever it is, explaining to it, because people are still really nascent. But that's also part of the problem.A lot of companies...this was in my Spanish class. If I started taking Spanish class for the first time, at the YMCA. And that was our topic, Chat GPT. And I'm like, no, I don't use it. Other people are like, "Yeah, I use it for this and this." But then the Spanish teacher who's quite...kind of identifies as a Luddite, he says he pays for Chat GPT because then he gets the license, then he gets the right to his own content that he could one day sell. And I was like, "I didn't think about that." I thought about it more because a lot of companies don't have generative AI policies yet, which is ridiculous.Look what happened to Samsung. We're recording this in early December, I think in September, a coder didn't think about it and checked like a whole code base live in the public, free Chat GPT feeding like a bunch of private information in. And now Samsung's like, no more, no more generative AI, we're done.ADRIANA: Yeah.JENNIFER: [inaudible] behind, instead of every company needs like law firms. People are using it for stuff at consultancies. But if you don't tell people, like, do not put public information in here, do not put IP in here, or just pay the $20 a month for Chat GPT. I think it's five a month for Copilot and it's just a much better experience anyway. So pay for your tools and advise people how to use them. So I think just super important because I just think it's clear that AI is just going to be a part of our lives.ADRIANA: It is, yeah. And we have to be more mindful of how we're integrating it in our lives.JENNIFER: Because what is it? Copilot went GA early June [2023]. It's early December now...maybe mid June. By the time of the Octoverse Report, which I think was early November, late October, 92% of developers in the US were using generative AI.ADRIANA: Damn.JENNIFER: We're testing out. Like you can't take this away. They are finding value from, yeah, you can't take this away anymore, but you really have to have a policy. And it's shocking how few do in California or GDPR in Europe. I'm shocked we haven't had a big problem. I'm shocked it hasn't been big yet.ADRIANA: Yeah, it's been sort of...as companies realize that it's important, they'll implement it into their policies, but there's like, no...JENNIFER: [inaudible] And putting really wild stuff. I have someone I know in the journalist space who is much more technologically advanced than I am and not a native English speaker. So they had put a very nascent new technology...had written like a really deep dive article, evaluating it, explaining tutorial. They had thrown it into public Chat GPT to clean it up. Then they delivered the client. Three weeks later, their exact article showed up on one of those clickbait sites.ADRIANA: Oh my God.JENNIFER: They can't contact an editor, because...they can't contact a human being, because it's a fake human being, because it's like a clickbait site. But that site had found that this new technology was trending and they trained that site in it. They trained Chat GPT in it. And then it just took out their article.ADRIANA: Damn.JENNIFER: Don't put stuff that's not published or public in a public AI, whether Bard, it's Bing, whether it's Chat GPT, you don't know what's going to happen. Pay for it. If you want to play around with it, maybe. But even playing for fun, it still has an environmental impact that no one seems to care about.ADRIANA: Yeah, I'm so glad that you're bringing that up, because the more we talk about it, I hope the more it gets into people's brains that we cannot take for granted the things that we use. I mean, even Google, right? The fact that you're googling stuff, I mean, there are servers running things somewhere.JENNIFER: Google tends towards green energy more than the largest one, AWS. Leslie Miley, who was speaking as himself, but does work at Microsoft, at QCon, gave this wonderful in his keynote, just a really impactful talk. And he analogized the growth in AI to the US and maybe one of the world's largest infrastructure projects, which was the interstate road system, which specifically created red lines, which specifically was like, strategically kept people of color from being able to use buses to enter New York City and work, which still to this day in San Francisco or that area, the Bay Area, where we have all this, I assume is the most inequitable place in the world, where kids are three times more likely to have asthma, severe asthma, by six years old because of where these roads were built. So this idea, and it's happening again with the access to electricity, the access to data, the pollution, the access to clean water, because that's what's used...water is being used to cool data centers and it's happening around the same lines and stuff. It has this ability to create this great inequity and without diverse people and thought on your teams, people aren't considering it. And we know, again, one of those statistics, just like happy developers are more productive ones, more diverse teams are more innovative and profitable, but we've got our masks over our eyes again and not thinking. And that's where we are.So sorry to end on a bummer of a note, but let's think of the...I'm always back to there's a wonderful, Agile practice called Consequence Scanning from Emily Webber and Sam Brown. And I just recommend just doing a consequence scanning sometimes. Thinking about it's just simple questions like if this scaled, who wouldn't be able to use it? What are the good intentions we weren't thinking about? And what are some negative intentions or consequences that could happen because of this tool? This is one of those things with open source that even more because if you're being truly open source, your code could be used, I don't know, making another Kiwi Farms or another hate site. Hate farm, that's the consequence of open source. You need to think early on, "Okay, what if someone used this for evil?"ADRIANA: Yeah.JENNIFER: Negative consequences or what are the environmental consequences?ADRIANA: Absolutely. And I think that's really great food for thought. And I hope folks who are listening to this really take this to heart. And next time they use a tool like Chat GPT, they think about the environmental impact or even when they're using resources on the cloud, think about these things because it's so important and we've only got the one planet and time is ticking.JENNIFER: And don't trust the news. Like, these jobs like mine as a tech storyteller are not going away. We need more people. We need more people explaining in different ways, in different languages and different jargon so everyone understands what is being built and why and what the consequences are. Because a lot of people are just using.ADRIANA: Yeah, absolutely. Well, thank you so much, Jennifer, for geeking out with me today. Y'all don't forget to subscribe and be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time...JENNIFER: Peace out and geek out, y'all.ADRIANA: Geeking Out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.

About our guest:Charity is an ops engineer and accidental startup founder at honeycomb.io. Before this she worked at Parse, Facebook, and Linden Lab on infrastructure and developer tools, and always seemed to wind up running the databases. She is the co-author of O'Reilly's Database Reliability Engineering and Observability Engineering, and loves free speech, free software, and single malt scotch.Find our guest on:X (Twitter)LinkedInFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:Honeycomb.ioThe Four Tendencies, by Gretchen RubinChoose Boring Culture, by Charity Majors (blog)Helicopter Management, by Charity Majors (blog)Choose Boring Technology, by Dan McKinley (blog)The Advantage, by Patrick LencioniQuestionable Advice: "My Boss Says We Don't Need Any Engineering Managers. Is He Right?" by Charity Majors (blog)Performance Improvement Plan (PIP)The Engineer/Manager Pendulum, by Charity Majors (blog)The Hierarchy is Bullshit, by Charity Majors (blog)OktaCharity's Calendly for career adviceParse, Inc.Honeycomb Pollinators SlackDevOps Research and Assessment (DORA)OpenTelemetry specification has gone GAAdditional Links:Observability Engineering (book)Database Engineering (book)Transcript:ADRIANA: Hey, y'all, welcome to Geeking Out. The podcast about all geeky aspects of software delivery. DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And geeking out with me today...I am so excited to have Charity Majors of Honeycomb on! Welcome, Charity.CHARITY: Yay! Thank you for having me, Adriana.ADRIANA: I'm so excited. And where are you calling in from today, Charity?CHARITY: San Francisco. I just got home. I was in Charlottesville, Virginia, with my little sister over Christmas, and so I am newly home again, looking forward to a very quiet week between Christmas and New Year's.ADRIANA: That is always the best week for chillaxing, right?CHARITY: Nothing going on. This is why at honeycomb, we just give everyone the week off. Obviously, some people have to be on call, but why pretend you're getting stuff done if you aren't?ADRIANA: I know, right? Yeah, I fully support that. I totally agree. I think more companies should embrace that.CHARITY: Yeah. I don't feel like anyone should have to be performing that they're excited to be at work or like, we don't make people have a set number of vacation days or anything, but...That's the worst. If you're like, well, it wouldn't really be working, but do I spend one of my precious vacation days? Yeah, fuck it.ADRIANA: Yeah, I agree. Honestly, I get so much anxiety over vacation days, like, having to meticulously plan them and, like, oh, where do I spend them? And maximize vacation with family and school holidays. And there's, like, so many school holidays, right?CHARITY: Seriously, there's no perfect system. Like, if you do the unlimited holiday thing, people are like, well, but then you're not treating it like real comp. And people have stress about, are they hitting the right number of days or not? And people won't take it. But then if you have specific number of vacation days, then it's where do I spend it? And everything. So I guess if there's one thing that being a CEO CTO of a company has taught me, it's that people are going to complain no matter what. All you can try and do is pick what is genuinely best for your people that will really help you get as much work as possible done without asking people to fake it and do a bunch of. So, we've gone the infinite vacation route, because, all things considered, I think you kind of want to have a mandatory minimum. Like, you have to take two weeks off, right?ADRIANA: Yeah.CHARITY: And above and beyond that, it's like, are you getting your work done here? It's a standard. The company standard is about three weeks a year, but nobody's looking over your shoulder and policing you.ADRIANA: Yeah. See, I appreciate those policies, especially at companies where they fully respect autonomy, because there's the companies where it's like, well, it's unlimited, but we really only expect you to take like three weeks or four weeks or whatever, and it's like, so it's not really unlimited. Right. And that's disingenuous and annoying and very stressful. I don't know. I bust my ass and I need the time to chill.CHARITY: Yeah. But I will say some people will start taking five weeks, six weeks. But then the question that you have to ask them is, you're taking too much time. It's like, well, are you really getting your job done? And what's the impact on the people around you? Really?ADRIANA: Yes.CHARITY: Because, yeah, it isn't actually fair if you take eight weeks off. Anyone would understand if you have a health issue or if someone in your family is. We've had those situations. But if you're working at a startup with some intensity, we have VC money that's burning in the bank. You kind of can't get your job done, really, if you're not there for two months out of the year.ADRIANA: Oh, yeah.CHARITY: I think always trying to steer it back to the impact. Right. Can you get your job done and are you letting down the people around you, or are you being a real functional member of a high performing team? Those are the terms to have this debate on not how many days you're here or not. The other thing, unlimited time, is that it removes the aspect of scorekeeping and time keeping and quibbling about hours, because some people don't really care, but some people get really concerned about, well, am I taking 2 hours off here and 3 hours there? If I take 4 hours of that a day or not? And those are brain cells that I would really rather you just devote to solving the problems that we're paying you to solve, not to bookkeeping around your own anxiety or your projected expectation of someone else's anxiety about the hours that you're spending on your job.ADRIANA: Yeah, absolutely. I have to admit, the timekeeping stuff is so stressful, and I've been lucky the last three years. I have not had to fill out any timesheets, which has been like, oh, my God, my first job out of college was, like, consulting. So all of your fucking hours are accounted for.CHARITY: Oh...ADRIANA: So everything and even your downtime, right? If you're in between projects, you got to charge it to internal thing. And it was like, yeah, I lasted four years.CHARITY: Oh, honey. I don't know how! One of our company values is we hire adults. And I actually think about that. It's as much about us as it is about the people we hire. It's like, are we treating people like adults? Do we expect them to manage their own time or not? And of course, the difficult points come. I think as an industry, we're just terrible at figuring out how to really take people on as apprentices and turn them into fully-fledged employees. I mean, there's that middle section that takes, even for a fresh college grad or someone entering...It takes five to seven years, I think, for you, really, to bring someone on and bring them up to a level of senior engineer and teach them all these things.But you can interpret it, our value as you're on your own. You better come fully baked because we're not going to help you, which is not what we're trying to project or do. But it's challenging, no?ADRIANA: Yeah. It's so challenging, like coming out of school, right? Trying to figure out where you fit in. And it's also kind of, for me, it was like a bit of a mind fuck because I was like the goody goody. Like, I will do all the assignments. And marks were everything. And then you go out into the real world and it's like, yeah, bye bye. That did not apply. For me, it was a massive adjustment and I kind of sucked fresh out of school, like my first couple of years in the work world trying to figure out, what do I do? What do I do? There's like, no marks. Not in the standard sense, right?CHARITY: No, of course not. You must be an upholder type. Do you get a lot of satisfaction out of checklists? Like your own checklists and the checklists that people do?ADRIANA: I do, I do. My own checklist. My whiteboard next to me. It's mostly clean now because of the holidays, but it had my to-dos...but I've had to learn to roll with it. I had to be a lot less uptight than I was in school, because I think you just have to, in the work world.CHARITY: Well, because you learn eventually that if you want to be successful, it's not actually about checklists, it's about figuring out what matters to you and what matters to other people and then figuring out how to creatively achieve those goals. And the checklists are there as a tool, right? I'm not telling you anything you don't know.ADRIANA: Yeah, exactly. Yeah. I completely agree. And I think that's a lesson that comes so much more easily for some than others for sure. Especially. I've hired a couple of interns in my past life and trying to steer them in the direction of, like, chill. Let's relax. Let's just focus on getting the work done and learning cool shit.CHARITY: In a lot of ways, though, I would argue that the upholder is the easiest type of person to onboard because they're motivated by everything.ADRIANA: True.CHARITY: So when I use the term upholder, I don't know if you've read the book, "The Four Tendencies"? It's this book that it's super cheesy and I don't want to get anybody's expectations up, but it was actually really pivotal for me and Christine [CEO of Honeycomb] and finding a way through our relationship because she's an upholder. I'm the opposite. I'm a rebel. Which means that I reject all of your checklists and my own too, called checklist. Basically, it's about motivation. And there's only four possible types.It's a two by two, right? It's like your own motivation, like what motivates you and the goals that you set for yourself and then the goals that other people have for you. And you can either be super motivated by both or you can be what's called a questioner type, which you can't really give a fuck about other people's expectations. But if you care about something, then you can hit that goal every time. And then there's the type that needs a gym buddy because you struggle to do the things that you set for yourself, but you respond really well to external structure. And then there's the type that rejects all of the structures. And that's my type. And this was really helpful to us in just like, sort of because Christine and me are just such polar opposites that she was just like, who the fuck are you? How does your brain work? Why is it that I give you this perfectly formed challenge and you're like, "Fuck all your challenges." And I'm just like, "Why are you telling me what the fuck to do? Don't you know that's the easiest way to demotivate me, is to tell me what to do?"And so it was really helpful because this book actually has these almost, like, examples of, if you're this type in a relationship with this type, here are some conflicts and conversations that you might have if you're in a working relationship and you're this type paired with this type. And it was just like, oh, my God. Some conversations that I had had with my partner, like almost word for word, some conflicts Christine and I had had, almost word for word. It was just like, here are some tools for getting around them. So I really like it.ADRIANA: That is so helpful. It's funny, because I think the way you describe yourself is how I would describe my daughter, too, to a certain extent, because when she was in preschool, her teacher could not teach her, and she realized that the way to teach her was not to teach her, but to teach her friends. And then it would cause Hannah to go over, oh, that looks interesting. So she's like, don't tell me what the fuck to do. I'm from Brazil. And I'm like, oh, it'd be so cool if you learned Portuguese. She's like, "No." What did she do? She learned German.CHARITY: That is how you deal with rebels. You have to rely on them to find their own intrinsic motivation, because if it becomes part of their identity and part of who they say that they are, then you can't stop them.ADRIANA: Yeah, exactly. Yeah. So I'm like, you know what? You do you. I embrace that. And I think she's happier for it. I'm happier for it.CHARITY: Everyone should be happier for it. As a manager, part of what you have to do is, I feel like, as a manager, in the beginning, we try to give our reports the experience that we wish we had had. For upholders and for...I can never remember...the obligers. Obligers are the ones that need the external structure. You're really giving them a gift. If you give them a structure or if you give them regular check ins and you let them know what the expectations are, you're giving them a huge gift, and they will rise to the occasion and they'll thank you for it. And if you do that for rebels or questioners, you're insulting them.That sort of versatility. And it's not just managers, of course. It's anyone who's, like, in a senior plus position, where what you need to do depends a lot in influencing others. Just sort of having a mental map of how other people respond to sort of motivations is super helpful.ADRIANA: Yeah. I actually remember reading one of your blog posts on, like, I think you're talking, like, being manager and trying to make everybody happy, but it's not also about being their buddy and making everybody happy, but also, you do have company goals to fulfill. And so to what extent do you protect your team, but then don't end up doing the things that need to be done, which I think is such a common pitfall for new managers, because for me, certainly when I first got into a management role. I'm like, this happened to me.ADRIANA: I'm not going to let that happen to my direct reports. I am going to be the best manager that I can possibly be. Right. It can kind of blow up in your face if you're not careful. Like, I wanted to be friends with my direct reports. That did not work out in the long run. Initially, it was like, yaaaay. But afterwards, it was like, no.CHARITY: We're always overcompensating for our own experience.ADRIANA: Yeah, exactly. And in the end, I think we learn, right?CHARITY: Yeah, exactly. Eventually, hopefully, we find a happy medium. I think about that so often when thinking about diversity issues in the industry or about management or that it's natural for there to be like, this is a young industry. This is a very young profession. For as old as some of us feel like we are, we're still like, there's been... When I was coming up, we didn't talk about women in tech. There was a few of us that were just, like, quietly there, wearing men's clothes and just sort of pretending we were straight white dudes. And so there was a lash, right? And then there was a backlash.And it swings. I'm not going to say too much about how sensitive I think some people are, but I understand why they are. I understand why they are. And also, that's not where we have to end. That can't be where we end up. We have to end up in a place that is less reactionary on all sides.ADRIANA: Absolutely.CHARITY: The goal of our businesses and our companies, this is something I've been thinking about a lot. The few times that I feel like the honeycomb culture has gone off the rails a little bit, is when we've kind of lost sight of the fact that we are here to serve our customers. We are not here to have the most diverse company in the world. We're not here to give people the best work life balance. We aren't even here to give everyone the best employment experience of their lives, which early in our, when it seems for so many years like we were going to fail, Christine and I would console each other. We'd be, you know, if we go under tomorrow, as we think we probably will, at least I think we've done a good job of giving a lot of people an experience that will set the know so they won't accept shitty jobs for the rest of their life. But now that we're hoping to be around for a long time, we can't forget that we are here to serve our customers. The decisions that we happen to think that a lot of these things go in harmony.Treating people really well means we treat our customers well. Having people who are happy at work. We believe in having healthy businesses, which is a lot of people's complaints. They see symptoms, but what they'reacting to is the fact that the business is not healthy. The way people are relating to each other is not healthy. I wrote this other blog post a while ago, I don't know if you saw it about, "Choose Boring Culture"?ADRIANA: That sounds vaguely familiar.CHARITY: You know, because Dan McKinley wrote that blog post that was hugely influential on me about choose boring technology where he's like, you know, as a startup you get three innovation tokens. Choose wisely. And I feel know the same is true for culture and businesses. And like, we stand on the shoulders of...you know, a lot of people, a lot of really smart people have figured out things about how to make companies work well. There's this great book by Pat Lancioni called the Advantage, which I think of as like the James Madison of business and organizational structure. He's incredibly innovative thinker and he makes things very simple. But he's like, the advantage increasingly in corporations is not your widgets. Because everybody's widgets are getting so good. It's how healthy is your organization, which means how much of your people's creativity are you really taking advantage of? How much of their creativity do you feel free to bring to work? Is your organization equipped to absorb it and to change from it and to react to it? Are you able to keep people who are passionate about their work? Do you let people go who are detracting from the culture? And he's like, it is amazing how poorly most organizations are run to this day.So choose boring culture. I think in a lot of ways, companies don't have to make their companies interesting and fun because people will do that. People have so much fun, creative energy in themselves. You just have to create a boring place for them to work where they can do their best work and they'll come up with all the fun stuff.ADRIANA: Oh, I love that. That's so cool. You touched upon something that I am a huge proponent of, which is like, letting go of people who are not adding to your corporate culture. Because I think there's this tendency, I think, in our industry to hire rock stars and kind of ignore the shittiness and their personality because, oh my God, they're the best of the best at blah. Right? And I've personally experienced a couple of incidents in my life where if you have somebody who is constantly just being negative on your team, no matter how good the rest of your team is if they're like, poo pooing everything, it sullies the culture. It's like a poison pill. And it's not like, oh, I'm going to fire your ass. It's like, well, perhaps this team might not be the best for what you want to achieve. Perhaps I can help you find a position in another team in the company. Because it's just poison.CHARITY: I think it starts with not having kid gloves on. I don't think you jump straight to firing. I don't even think you jump straight to moving. A lot of these people have never really been told no in their lives. And some of them can take it, some of them can. But I think you owe it to them to figure it out, right? To start giving feedback consistently and regularly working with the person. And this is something that I think can be really frustrating to people who are. When it looks like management is doing nothing right, because it looks like, I know that people at Honeycomb have felt this way at times, because it looks like they're just kind of being shitty and they get better and then they don't.And it's always a judgment call. And I would actually agree that we always probably wait a little too long in general, but we waited a little too long with everyone. And I would take that over being a little too fast to fire people, because I think that that even more trust. But, yeah, I agree. If they can't bend, if they can't change, if they can't understand that the smallest unit of software ownership is the team, it's not the person. It doesn't matter how great one person is, because one person can't own software. It's all about, are you contributing to the overall greatness of this team? You can bend your rockstar talents to that, but if you're not willing to, or if you can't, then there's no place here for you. I'm sure you can get paid a lot more money somewhere else.ADRIANA: Yeah, absolutely true. Absolutely.CHARITY: Sorry, go ahead. I didn't mean to cut you off.ADRIANA: Oh, no, I was just saying I agree with you, but I think that.CHARITY: Letting go of people is hard, and I think that it comes in all forms. I think that it's really discouraging to people who are on a high performance, who want to be on a high performing team, when someone isn't really showing up and who consistently isn't showing. The person who's like, consistently taking six weeks of vacation when everyone else is taking three or four, or the person who is kind of half asking it. And all of us half ass it sometimes, right? But people can tell you work on a team for a while, you get a real good sense of how hard everyone is working, how much they're trying. Sometimes it comes in form of, this is almost some of the most heartbreaking ones of when you've got someone who's very junior who just isn't working hard enough. And it's like we kind of don't have the language to tell them that. Because on this pendulum, we're so far over to the side of, you shouldn't be like, work crush code. It's almost like we've kind of lost the ability to tell people, no, really, you're probably not going to make it if you don't put in a few more hours and if you don't have a little bit more grit.And some people don't want to work that hard, and that's fine, but you aren't automatically granted a job based on however hard you do or don't want to work.ADRIANA: Yeah. And it's such a tough conversation to have. I had someone in a previous team that I hired on as a senior person, and then she was, like, scamming on my. She was scamming on everyone else. She would just pretend that she was doing work by, like, oh, let me attend meetings with so and so. And meanwhile, I'd hired this junior person who was working like she was working at the senior level. And it was so frustrating. I was trying to have the conversations with the senior person saying, listen, I want to help you. How can we work together? But she got offended. And these conversations are so hard to have because we all perceive differently how we're doing. And in her mind, she was doing just fine. How have you had those conversations in the past with people?CHARITY: Oh, it's really hard. There's no version of this that isn't hard if you care about people.ADRIANA: Yeah.CHARITY: My most recent blog post was about why anyone should go into engineering management. Because it's a hard fucking job. And the answer is, because we need them. Because we need them desperately. Like a team with a great engineering manager builds circles around teams without one. And the other reason in my piece, I said is that it changes you as a person, and it gives you these skills that a lot of us didn't learn when we were growing up about how to be honest and how to have hard conversations and all these things. But as to your question, how do you go into this? The number one thing I think is no review should be a surprise. You should be having this conversation consistently, which is a hard thing to do because it makes people feel demotivated and frustrated.But sometimes they have to feel that way. We've instituted a rule at honeycomb that if you're thinking of putting someone on a PIP, if you're thinking of, you have to literally say the words, your job is at risk because it's so tempting when you're face to face with someone who you really want to succeed, to soft pedal it or for them to feel upset and for you to kind of walk it back, or for you just to use words that let them walk away thinking something that is not what you want. And there are tools you can use to make sure. You can write up an email afterwards to be like, just to be clear, this is what I saw. This is what I'm saying. This is what you're hearing. But I really do think that one of the most important tools we have is just being explicit because they can file it away. We all have such infinite creativity when it comes to explaining away things that we don't want to hear.And we can be like, oh, my manager is kind of a bitch. Oh, they're just in a bad mood. Oh, they're just kind of riding me lately. Oh, it's because of this thing. But this will be over. And I feel like if something really isn't trending, well, we have a responsibility to be more of a dick. We have to be the ones who kind of put our bodies in the breach and be like...and just sit there and deal with their reactions, which are going...They're going to have negative feelings. And it's really hard to sit with someone else's negative feelings who you are the proximate cause of. It's really hard, but you have to do it. It is the best thing for them to do it, to let them know this isn't just a small thing. This isn't just a flash in the pan. You are not succeeding. You are not on a path to succeeding here. You are on a path to, your job is at risk. Honestly, that's the kindest thing you can do for someone.ADRIANA: Yeah, that makes so much sense. And you're right. It's so hard to get those words out. Like, "Your job is at risk." Yeah. And I've worked in organizations, too, where pussyfooting around the topic was like kind of the cultural norm, and so things wouldn't get said that should have been said, and you don't have the favorable outcomes in the end.CHARITY: Yeah. And then people feel stabbed in the back, understandably. I would, too. They go...walk away going, "If they had just told me, if I had only known." And that is the worst outcome. That is the thing that I always remind myself of when I'm just like, I love this person. I don't want to be mean to them, but I cannot take it if they walk away feeling like I didn't tell them, like I stabbed them in the back by not making it perfectly clear that they're not performing and their job is at risk.ADRIANA: Yeah, it's definitely something that I wish that I had done more of in the past, and I try to remind myself of it, but, yeah, I think that is absolutely the right thing.CHARITY: And to your point earlier about being people's friends, you can absolutely be friends with your direct report, but there's a line there. There's a boundary there, and there's a point at which you're not their friend. It's just like being someone's parent, right? When things are going great, yeah, you act like friends, but they have to know that when it's time for you to be parent, you're going to be parent.ADRIANA: Yeah, exactly. Because otherwise they will take advantage of you.CHARITY: Right. They will completely take advantage of you. It's human nature.ADRIANA: Exactly. And you will let your guard down, too, right? Because they're like, oh, "I don't want to hurt so and so's feelings, otherwise they won't love me." And it's like, you kind of have to get over that as a manager. And it's hard.CHARITY: It's really hard. It's really hard. And it's always a matter of judgment. It's always a judgment call. And you have to know that after you've had that hard conversation, chances are they're going to go tell other teammates a version of it that makes you look bad and them look great. And you can't do fuck all about it. You have to sit there and take it and hope that the relationships and the trust that you have built up are enough that people aren't going to just automatically believe that other person. That is the hardest thing about being a manager to me.CHARITY: That right there, knowing...is when I know I can't say anything.ADRIANA: Yeah. And risking, as you said, having people say, well, management doesn't know what they're doing. Oh, my God. Because as an IC in the past, I was like, management clearly doesn't know what they're doing, and then...CHARITY: Clearly doesn't know shit.ADRIANA: The first time it happened to me, oh, my God, I want to go cry. Like I'm trying everything to make you happy.CHARITY: Yeah. This is why I feel like my dream vision for the future of engineering management is that more people do it. But people don't do it. They don't do it as a career. They do it as a tour of duty, because I feel like having ex managers on the team, it's like a game-changer, because whenever the dynamic is ICS versus managers, which always happens. Comes and goes, but it always happens. It's so helpful to have an ex-manager there on the IC side who could go, okay, kids, it might be this. It might be this. It might be this. Do we trust this manager in general? Okay, well, let's not jump to the automatic conclusion that they're just an idiot or they're just, like, being manipulated by the upper or whatever. They're the only voice in the room who can talk people down off a cliff and remind them whether to have some trust. And it's such a game changer. It is so wonderful.ADRIANA: Yeah, that is so true. And it makes so much sense. I even find myself in positions after I've been a manager, and then being now an IC...whenever I get comments...CHARITY: It's nice!ADRIANA: Yeah, it is nice! And sometimes I have my manager apologize, "Oh, I'm so sorry. Blah, blah, blah." I'm like, "Dude, I totally get it." "It's fine. No worries."CHARITY: You're able to give so much better support and understanding to your manager than you ever could have without that experience.ADRIANA: Exactly.CHARITY: It's so grounding and validating for them to have someone who sees them.ADRIANA: Yeah. And especially, also when you have that nice rapport with your manager where you have that ultimate trust, where, okay, it might seem like they're riding you hard, but then you're like, oh, my ex-manager brain has said, okay, "I have a good reason to trust them. Take a step back. Let's look at the big picture." And, yeah, it's cathartic and it's eye opening.CHARITY: Everyone wins.ADRIANA: Yeah, exactly. No, sorry. Go ahead. No, please.CHARITY: I often hear people who are first-time managers who are, like, anxious or like, if I go back to being an IC, will I ever get the chance again to be a manager? And I'm just like, "Oh, grasshopper, they can smell it on you. You will be fighting off manager opportunities for the rest of your career." Have you found this to be true? I expect you have.ADRIANA: Yeah, I have. And it was funny because after I read it in one of your blog posts, I was like, oh, yeah, so true.CHARITY: Yeah.ADRIANA: I mean, it's on your resume. Yeah.CHARITY: Just the way you come across. I've also said that the fastest way to mint like, a shiny new staff engineer is to take a senior engineer and put them in management for a couple of years. Because the way you present yourself at work, the way you approach problems, you have such a better sense of the business, even if it wasn't on your resume. This is why some people get to be managers early and often, because for whatever reason, they already have some of those skills. But once you've been a manager, it's written all over your face that you understand.ADRIANA: Yeah, very true. Now, here's a question for you. What's your take on folks who have gone into management at a really early point in their career, becoming a technical manager for a technical team when they don't have that many years of actual technical experience?CHARITY: I think they are not well-served by this. I often see this happen to women, especially, and I think it's often intended as a compliment and by people who genuinely are trying to do they want to help the industry. They know that there needs to be more women in leadership and management. And so they're like, here's this person who has social skills and also some engineering skills. So we'll just...I think everyone has the best of intentions, and I think it really does not serve them because it's often a one-way...it's a one way-ticket, right? Because you don't have the skills to be able to go back and pick up coding easily in a couple of years. I think you also don't really have the skills to be a great manager.Honestly, my recommendation to them would be get back to coding as quickly as you can or climb the ladder. If you choose to climb the ladder, then those skills are less relevant. But I wouldn't be in a rush. If you're 25 and you're a manager getting offered a director position, I would look at that cross-eyed. I would be like, because, yes, it is probably a compliment, but is it the right thing for you? I don't know. I mean, if you play out over the course of your career, you've got a 30, 40 year career. There's no rush. And the people who really excel in those senior leadership positions tend to be ones with deep roots, not just a very shallow.And there's so much to learn, right? This is not to say that there's not anyone out there who's climbed the ladder in a hurry and not regretted it, because there probably is. But the people that I know who have done it have, by and large, profoundly regretted it. You know, I wrote about my friend Molly, who's an engineer at Honeycomb now, and she was one of those people. She super bright, straight out of college, became an engineer, became a manager, became a director. Shot up. You know she was a VP, she was a director, she was an EP. And she came to Honeycomb to be our head of...VP of customer success or something like that. And she was so unhappy.And she would make all these wistful comments about how she wished she could be a software engineer. She wished she had done that. Eventually, her husband, he was an early member at Okta and Okta IPOed. And so suddenly she was like, "Wow, I can do anything I want with my life. I want to be a software engineer." And so she became a support engineer for us, and she just started writing code on the side. She started picking up some PRs. Now she's a software engineer on the team, and it's been hard.She's never been happier, though. And I'm proud that Honeycomb is the kind of place that can support someone in doing that, because I think the opportunities to do something like that are few and far between. There are not many places we'll take a flyer on someone who's middle-aged and wants to go back to software engineering. But if you think of your career as a long game, you don't want to amass a bunch of titles, especially titles that are kind of empty because you're not getting a...I would...I would venture to guess that you're not getting a really high quality offer to be a director or a VP at age 27. It's really mostly the title. You want to amass yourself a solid base of experiences and skills, and you want to have shit to draw on as you climb that ladder so that you can help people better.So the thing that I do want to guard against when I'm talking about this, I'm speaking to people who are early in their career, who are facing these questions. I don't want to make it sound like it's too late and you're screwed if you're already in this position. In fact, if you're in that position, if you'd like someone to talk it through, reach out to me. I have a Calendly link, calendly.com/charitym/advice, and I'm always happy to talk through interesting and tough career conversations with people. You have skills, you have assets. It might not be a super sexy path, but you can find places that will take advantage of the skills you have to offer while you kind of work your way up from the bottom again, if that's what you want to do. I'm sure you can do it, but it's easier if you do it right the first way and become a solidly senior engineer. Seven years really is the minimum, I think, before you become a manager.And if you really want to be able to manage other senior engineers, you need to at least be able to speak the language and be able to roll back on it.ADRIANA: Yeah, I fully agree with you on that. I was thinking back to my own career. My first job out of school was as a consultant at Accenture, and the career path was basically like, you must pay your dues as a developer, and you shall be rewarded with a management position. Right? Yeah. Right. So we're all kind of brainwashed to think, oh, my God, if I'm not a manager, by 27, 28, I have failed at life. Right? And I hit this crossroads in my life where I was being groomed to be a manager. I didn't have the manager title, but they threw me on some engagement where I was managing three teams at once. I was doing a shitty job, and I'm like, I was miserable, and I'm like, what do I want to do with my life? And so I decided...I left consulting. I took on a job as a software engineer. It was a lateral move, but I was so happy, and it was the best thing for me because my thought was, how can I manage these people if I don't know enough? I just didn't feel right for me, so I'm happy I did that.CHARITY: Good for you for listening to your gut. I think all too often we talk about impostor syndrome, and we try to talk people out of it. I often think if your gut is really eating at you, that something is wrong. You should listen to that. You shouldn't just go, oh, everybody, there's impostor syndrome, and then there's just, like, the feeling in your stomach that you're not really setting your future self up for success or that you aren't really equipped to do the kind of job that you want to be able to do in this role. And I think that is not something to be brushed aside lightly.ADRIANA: Yeah, I definitely agree. Listen to your gut, because it's telling you something. One thing that I wanted to ask was, when you were building Honeycomb from the ground up, did you have sort of lofty aspirations of how you wanted things to be?CHARITY: Ha!ADRIANA: How was...the initial thoughts versus how it turned out?CHARITY: I 100,000% expected us to fail like the plan was to fail. So I was never one of those kids who was like, I'm going to start a company. Because I always kind of low key hate those people. It's like, "Oh, you're too good to work for someone else." I'm not too good to work for someone else. I was a serial dropout. I'm the opposite of you, right? I didn't collect all the awards. I didn't check anything off.I dropped out and I dropped out again. I dropped out again. And so I never had a pedigree. Nobody was ever going to give me money. Then I was leaving Facebook, and the first time in my life, I kind of had a pedigree. And so I was like, well, I can't waste, like, on behalf of all women and queers and dropouts everywhere, I have to take it and run with it and do something. But I was super burned out. And I was like, well, I guess I have an idea, but I'll go heads down the corner, write code for a couple of years, and then we'll fail.And I'll open source it. Then I'll have my tool to use. Hee haw! That was really the grand vision. And I would say Honeycomb has been around for eight years as of January 1, but we had many near-death experiences. Now, we hit our $40 million ARR mark this month, which is exciting. We're hoping to get on a path towards an IPO. But for the first five years, I think we wobbled around between 5 people, 12 people, 30 people. We did layoffs down to 15 people again. We were a skeleton crew wandering in the wilderness. In retrospect, I realized that we were creating a category and we were writing the database and all this stuff, but it just felt brutal. It just felt like failure was around every corner. And most of those corners were right. We did fail most of those corners. There are several just, like, near-death experiences that we had, and we made it through.And now I, for the first time, am not thinking we're going to fail. But no, there was no grand vision. There was no grand vision at all. There was just, like putting 1 foot in front of the other and feeling like I was failing the people that I loved most almost every single day. It was brutal. I will say, though, that Christine and I, a little bit older than your average tech founders, especially me, and turns out we have very strong opinions. And we learned a lot of lessons at previous startups. We were at, like, at Parse, which I loved working at Parse.Parse is where I learned about the importance of design, about marketing. People loved that product and I loved working on it. Before Parse, I was like, I'm just a backend engineer. I don't care what the product's about. I'll work on anything. Parse is where I learned that, of course, that was never true. But Parse never really had a shot because the founders never really tried to build a business. They tried to build a great product, and they did. But then around series B, they had a marketing person and a couple salespeople.We weren't bringing any revenue. They had to sell. Their destiny got taken out of their hands because they had no other choice. And so Christine and I, from the very beginning were like, we want to build a business. We want to build a business. We want to build a product that people want to pay money for. We're not building freebies. We're going to try and monetize on the other end of the pipe.We are building a product. We're building a business. And I had a lot of just, like, very strong opinions about the kind of culture we wanted to build, just about how...in the beginning, when we were interviewing engineers, if anyone talked, not even dismissingly, about go to market functions like sales or marketing, even just sort of, like, almost alienated, just like, "Oh, well, that's them. We're us. We don't understand that." Those weren't our engineers, because we don't need to hire engineers who wanted to build a business with us and who weren't going to create that us versus them dynamic that makes all great business people in the valley feel like second-class citizens. So, yeah, I would say we discovered the grand vision along the way. It really wasn't there from the beginning.ADRIANA: And as a follow-up, you know, one of the things that I admire so much about Honeycomb is you build such a lovely community around your product. Your customers truly, truly love it. And we met because I was asking so many questions in the Honeycomb Pollinators Slack. At the time, I was exploring Honeycomb as a potential product that the company I was working for might switch over to. And everyone was just so genuinely nice in helping me understand this Observability thing that was so nebulous. How do you build that thoughtful community? Was it something that you sought out to do from the get-go? Is it something that organically grew?CHARITY: If you ask any founder, they'd say they're trying to build that, right? So I think the questions were like, "Why were we more successful than many others?" I think a lot of it has to do with just...and if you had asked me if I would be talking about values and shit, like a year ago or a few years ago, I'd be, like, rolling my eyes, because I've always hated when people are like, "Values," because most businesses are just like. I don't know. I get really cynical about it, but I feel like we are our customers, and our customers are us. We built this product to solve a real problem that we are having. And it is more important to us that these problems get solved than that Honeycomb is successful. I think I can say that about everyone there.We would love to be successful. We'd love to make lots of money and all this stuff. But we see the pain that so many teams are in, and we know that we have a way to fix a lot of that pain, because we've seen our customers do this over and over, and we hear what they say about how no one else could do this. And we had the advantage of designing and building this 25 years after metrics began dominating the landscape. So we build on the shoulders of giants, like I said earlier. So I feel like it's easy to be a true believer, because we're not just trying to sell something. We're really building something that really changes people's lives. And it's easy to get starry-eyed about that.It's easy to be a believer when you're all on the same page about fixing problems, not just about trying to tweak your messaging or your marketing or your sales or something. I think people, Honeycomb, are generally very passionate about solving the problem, and it's very exciting to see them. I mean, the product does what it says on the sticker, which is very exciting, because almost no products do. Most products are hyped. If anything, Honeycomb is underhyped. It does so much more than we've been able to explain to people, which is why our churn is like nothing. We win, like, 80% of our tech evals, which the industry standard is, like 30 or 35%. Once people see it on their data, you cannot pry it out of their cold, dead hands.One of our best sources of leads is when engineers change jobs and they bring us with them, because once they've tried developing with Honeycomb, they can't go back to not having honeycomb. And this is all stuff that it's hard to explain to people in words, but once they see it, it clicks. And so, really, our core challenge, over the next year, we've built the product. Our core challenge is figure out how to get more people to click with it faster, because we know that once they've seen it. The deal is done, but it's still a very hard problem.ADRIANA: Yeah. The other thing that I think is very interesting about Honeycomb is it's not only are you building a product that people are excited about, but you've also really turned the whole area of Observability on its head. I'd like to think that it was Honeycomb that sort of gave Observability...Observability became what it is because of what Honeycomb has done. I mean, you've spent a lot of time talking about Observability. I mean, honestly, that's how I got dialed into what Observability was in the first place, was catching your Tweets. Yeah, if you could say a little bit more about that.CHARITY: Yeah. Like, Christine and I are not marketing people. It turns out what we were doing was category creation. All I knew was that we were trying to build something based on an experience we had had that had changed us as engineers, and we knew that it wasn't monitoring. And I spent months just sort of, like, testing language, trying various things. And one point, it was July in 2016 that I Googled the term "Observability", and I read the control theory definition, and I was like, "Oh, shit. This is what we're trying to do. We're trying to build something to let engineers understand the inner workings of a system, no matter what's happening, just by observing its outputs."So, like, working backwards from that, what do you need? Like, you need the high cardinality, you need the high dimensionality and all this stuff. And I feel like that definition really took hold for about three years. In 2019, 2020, maybe 2021, all of the money started rushing into the space, and suddenly, anyone who was doing anything with telemetry was like, cool. We do Observability, too, which, on the one hand, is like, it's a good problem to have. It means that what we were talking about really resonates with people. And at the time, I was naïve enough to think that, oh, well, they're co-opting our marketing language, but surely they're building the same technology under the hood. It's just a matter of time until they release it. I don't believe that anymore.I think all they did was steal the marketing language, and I don't think they actually have any plans to. I think that, like, Datadog in particular, their business model is centered around having all these different SKUs, right? A different product for metrics, for logs, for tracing, for profiling, for security, and they've got too much money invested in. The problem is that the experience degrades for everyone if nothing connects all these data sources. People are paying to store their data again and again and again and again, but nothing connects it except the engineer who's sitting in the middle just trying to visualize or visually correlate. If that spike is the same as that one, it's fucking broken. My hope is that there will be new startups that are entering the space. So I've kind of given up like, okay, Observability now means, and this makes sense, I'm actually completely on board.Observability, instead of having a strict technical falsifiable definition, Observability is a property of systems, right? A system can be more less observable if you add some metrics, great, you're more observable. But what we're seeing in the field is that there's a real huge step function difference between, let's call it Observability 1.0, which is about metrics, three pillars, right? And Observability 2.0, which is based on this single source of truth. And it's not just the technology, because o11y 1.0 is very much about MTDR, MTTD reliability, uptime. It's a checkmark before you send your code to production to make sure that it's observable. And Observability 2.0 is about, it's the foundation of the software development lifecycle. It defines your velocity, how fast you can ship, how well you can ship, the quality of what you ship, your ability to iterate quickly, your ability to identify what your customers are actually doing and why, and build on that. It's your ability to see what's happening in the wild and make decisions based on real data and then feed them. Because this is all about feedback loops, right? And it's about learning to be a developer where you're developing with fast feedback loops.And it's like the difference, o11y 1.0 is about, okay, this is something that you tack onto a product...2.0 is about, this is how you build the product, right? So many teams are stuck in 1.0 land and they're happy with the tools that they have, but the teams that are going to win are the ones that not only adopt 2.0 tooling, but also adopt the 2.0 mindset of this is how we build software. It's like putting your glasses on before you drive down the highway. You can drive a lot faster, you can make better decisions much more quickly. So I feel like right now, the big problem that Honeycomb has from a business perspective is that far too few engineering leaders even understand that 2.0 is possible because you can have a 2.0 mindset. But if you've only ever seen 1.0 tools, it's janky. It's real hard to like...you can only do so much, right? You really need to see 2.0 tooling in order to really...But it clicks so fast when you do. So that's really our job. For a long time, I was really disappointed that there are still Observability startups starting. They come up, ping, pong, like here and there, everywhere, but they're all 1.0 tools. They're still doing the multiple storage places. My hope is, and I get why, it's because you have to build an entirely different storage layer from the ground up. And very few VCs have the patience for you to do that. They want you to get right to product, market fit and all this stuff. Now that there are more columnar storage engines out there like Snowflake, I don't know...I'm optimistic, but I'm optimistic over the long run, our model of Observability will win. Even if Honeycomb completely fucks up in the end state is the complexity of our systems is increasingly demanding it. The complexity of people's systems is skyrocketing. You look at the DORA metrics, and I was always kind of like, dude, it's so weird. Like high performing teams, okay, that takes an hour to a day to restore service. But for the bottom like 80% of teams, it takes them a day to a week to restore service from an outage. How? It's because they don't have Observability.It's because they can't actually see what's going on. They rely on a few people's brains, people who've been there for a long time, who pack a lot of context into their heads, who can try and reason about it using the very limited data sources that they have. That's why it takes so long over and over. Part of the reason we win so many of our POCs is because over and over, our sales engineers, we help you roll it out, and they'll be like, is this an outage over here? We're seeing something wrong. And people will be like, what? Ten minutes later they get paged and they're like, oh, it's just like once you have this feedback loop, you get used to being constant conversation with your code instead of just like shipping and waiting for someone to get paged. At some point in the next hour two year, right. It's all about hooking up this feedback eventually, even if it's ten years from now, the model that we're talking about is the shape that's going to win whether it's us or not because our systems simply demand it. There's no other way to build software at that kind of velocity and scale.ADRIANA: I completely agree and I think having that conversation where Observability is considered...is baked into like...you're shifting left on Observability basically, right? Were it's like...CHARITY: Exactly.ADRIANA: No, it's not the thing that's tacked on at the end per usual. It's the thing that your developers are considering in the beginning that your QAs are using to troubleshoot shit and write trace based tests and that now your SREs are like, "Oh, I've got the information to solve the problem!"CHARITY: So many of the promises of Agile development and all these SREs and all of these cultural movements, they've never really lived up to their full promise. And I feel like the reason is because it's not just a cultural thing. You have to have the tools that actually make hard problems easy as well. And the feedback loops with metrics and logs are just painful and arduous and relies on so much on manual cross-correlation and heroes jumping into the break. But when you have the right tools, you can just glance at it and see the answer. And it's what unlocks the ability of teams to just be constantly...When I think about modern software development, I think about feature flags which help you separate releases from deploy so you can be deploying small changes constantly.CHARITY: I think about future flags, I think about Observability, just the ability to see what the fuck is going on at any point. I think about testing in production and I think about, well, canarying. There was one other thing that was on my mind. There's really just a four thing and they all reinforce each other, right? One of them alone is okay, but you get all of them together. And it's a completely different profession than it is in software development, which is kind of still from the shrink wrap era. It's like you're building, if your world while you're building software is your IDE and your tests, that's shrink wrap days. Your world should be production and telemetry. You should spend more time in your production windows than in your IDE windows. That's what modern software development is like I think.ADRIANA: Yeah, absolutely. And the final point that I wanted to touch upon is you mentioning...having...the data that correlates right? Where you're not just having to figure out how it's stitched together. And tools like open telemetry definitely enable that. But then I guess part of the irony though, is that open telemetry allows you to correlate traces and logs and metrics. But then if your Observability backend doesn't have a way to show that correlation, then you're kind of up a creek too.CHARITY: So I am so glad that OTel came out when it did so that I think we were able to have a lot of influence on how the data is gathered. You're absolutely right. Part of observability is the presentation of the information. If you don't have the ability to slice and dice, if you don't have the ability to combine, if you don't have that single sort of truth, then you can't really reap the rewards of Observability, even if you captured it. But capturing it the right way is the first step, for sure.ADRIANA: Yes, absolutely. And so glad that OpenTelemetry has gone officially GA. The specification has gone GA end of 2023. Long time coming. I'm super stoked for that.CHARITY: It's a big moment in our industry.ADRIANA: Yeah, and I'm so glad also that so many of the vendors have come together to rally behind it. And it's really not someone trying to flex their muscles over everyone else. It's such a lovely community.CHARITY: The only lagger is Datadog. People need to keep putting a little bit of shame and pressure on them because they're the only ones who are not playing nice, but everyone else is, which is a tremendous achievement. Huge kudos to Splunk, who's got like 30 engineers working on integrations every day. We would not be where we are without Splunk.ADRIANA: Yeah, it's so great. It's so great seeing all these innovations, collaborations, and people really genuinely caring for the project.CHARITY: It's great.ADRIANA: And on that note, we have come up on time. And thank you so much Charity for coming on geeking out with me today. This was awesome. One item off the podcasting bucket list for me. Always a pleasure to chat with you. And everyone, please don't forget to subscribe, be sure to check out the show notes for additional resources, and connect with us and our guests on social media.CHARITY: Until next time, peace out and geek out.ADRIANA: Geeking Out is hosted and produced by me, Adriana Vileela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout. My wonderful editor daughter will edit out any, any stuff. I pay her good money.CHARITY: How old is your kid?ADRIANA: She's 15.CHARITY: Nice.ADRIANA: That's a good age. Yeah. And she sports right now...she's sporting some really rad pink hair. Last year, she had gone purple, and I just took her to get a cartilage piercing, which I'm like, hey, I have no issue taking you. No issue taking you. I'll look away while it happens. Yeah, it's super fun. Super fun.CHARITY: I went to college when I was 15, and I felt very adult at the time. And now I look back and I'm like. I was a child. What was I doing?ADRIANA: You feel so old when you're in high school or like, when you're 15. I remember when I graduated college and I'm like, everyone looks like a baby.CHARITY: Yeah. Time of rapid change.ADRIANA: Yeah, for real.

About our guest:Kelsey Hightower has worn every hat possible throughout his career in tech, and enjoys leadership roles focused on making things happen and shipping software. Kelsey is a strong open source advocate focused on building simple tools that make people smile. When he is not slinging Go code, you can catch him giving technical workshops covering everything from programming to system administration.Find our guest on:X (Twitter)MastodonFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:CompTIA A+ CertificationRube Goldberg MachineHerokuKornShellCapistranoCloud FoundrySpring BootDistributed denial of service (DDoS)HashiConfMitchell Hashimoto (HashiCorp co-founder)Armon Dadgar (HashiCorp co-founder)Borg whitepaperSidecar (Kubernetes)Nomad on Kubernetes (GitHub)Hashinetes Talk (HashiConf 2017)From Community to Customers (KubeCon EU Amsterdam 2023)ConfdFOSSDEM (conference)Apache License, version 2.0RAIDWestworld LoopAdditional Links:Kubernetes the Hard Way (GitHub)Transcript:ADRIANA: Hey, y'all, welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And today I have the pleasure of geeking out with me, Kelsey Hightower. Welcome, Kelsey.KELSEY: Happy to be here.ADRIANA: And where are you calling in from today?KELSEY: I'm in Washington state, so on the border of Portland, Oregon, and Washington.ADRIANA: Awesome. Well, let us get to it with the warm up questions. Are you ready?KELSEY: I am.ADRIANA: Okay, first question. Are you a lefty or a righty?KELSEY: Right handed.ADRIANA: All right. iPhone or Android?KELSEY: iPhone forever. And I've tried android. Given that I've worked at Google for almost eight years, I've tried, but I'm an iPhone person.ADRIANA: Yeah, I'm an iPhone person too. I never tried android. I went straight from BlackBerry to iPhone.KELSEY: I think BlackBerry was definitely...I was a BlackBerry person. I was also a Nokia person. But I think once iPhone really dialed in the ability to have third party apps in the App Store, iPhone all day.ADRIANA: Yeah, I'm the same way. That was like one big sticking point. And for us in Canada, when the iPhone first came out, we didn't even have access to the App Store. So if you wanted any apps, you had to jailbreak your iPhone until it finally became available...because we get everything a little bit late here.KELSEY: Awesome.ADRIANA: Okay, next question. What's your favorite programming language?KELSEY: The one that I can get things done in. So, at one point it was Bash, then it was Python, then it was Ruby when I worked at Puppet Labs, and then it's been Goblin, probably for the last ten years.ADRIANA: Cool. Awesome. And Mac, Linux, or Windows?KELSEY: Mac on my desktop. Linux on the server.ADRIANA: All right, next question. Dev or Ops?KELSEY: They're one and the same.ADRIANA: I love it. Okay. JSON or YAML?KELSEY: JSON. If I had to program against it, YAML if I had to write it.ADRIANA: By oh, yeah, I definitely agree. I do find, like, manipulating JSON in Python is nicer, but YAML is more readable.KELSEY: Yeah. To all the people that are like, JSON over YAML, let me watch you write it and see how fast you change your opinion.ADRIANA: Yes, I totally agree with you there. Okay, this one's a little more controversial, and you can thank one of my previous guests for hinting at it. Spaces or tabs?KELSEY: I don't care. I actually don't care if Python makes me uses Spaces and my IDE does the right thing. I'm totally fine, actually.ADRIANA: I'm down for that. Okay, two more questions. Do you prefer to consume video or text when you're consuming content?KELSEY: It depends. If I'm trying to learn, I need to read it, I need to see it, I need to be able to kind of backscan read it twice. But I do like video in terms of when people are really good at the human side of it. Right? Like, if they're expressing or showing me something, like, I want to see the code run. I want to see where they click. I want to see how they start. But when it's like learning something in the programming world, I need text. People are pretty bad at video and programming lessons.KELSEY: Like, oh, just write these three lines of code. I'm like, can you please scroll up so I can see what you imported to make this work? So when it comes to seeing code, I want to see no snippets. I want to see as much as possible, but if I'm just going through for the first time to get the flow, video.ADRIANA: I totally agree with you. And you landed on one of my big pet peeves. When consuming content for learning stuff, which is the code snippets, because I have been and I'm sorry, Hashi people, but this is a crime on the Hashi docs that I see all the time is that I get code snippets, and I don't get to see a full example on the site, and it drives me bananas. And I'm like, what does this apply to? Give me a full fledged code example? Link me to a GitHub repo at some point.KELSEY: I'm always asking, why are people writing docs out there giving me hints to a murder mystery?ADRIANA: Yes.KELSEY: Show me the whole thing. I don't need it to be cute. I don't need it to fit perfectly in your style guide. I just need to see the whole thing and what's going on. So I think people do it out of style. There's really no substance when I'm trying to learn.ADRIANA: Yeah, I completely agree. I do find it very frustrating. That's why, for me personally, whenever I do technical docs, I give excruciating detail. All right, final question. What is your superpower?KELSEY: My superpower? I think one thing that I've learned over the years when it comes to mentoring, specifically, I used to be all about sharing my expertise, my background, my learning. And I've noticed that I changed my approach to holding up a mirror in front of other people and convincing them to like what they see and the number of people who actually like what they end up seeing and follow up with me. I really felt like that is a superpower, that you can actually have that impact on people. So that would be my superpower.ADRIANA: That is such an incredible superpower. And I think it's so relevant to our industry, too, because we have a lot of smart people who suffer from impostor syndrome. And I think showing people that you are actually as good as you think you are is such a huge thing. Right? I mean, we've got some amazing stuff happening. I have some coworkers who are brilliant, and they're like, oh, my God, I feel like I'm just a hack. I'm like, Are you kidding me? I can't even keep up with some of the stuff that you're telling me right now.KELSEY: Yeah. And I try to get people to understand that sometimes you aren't as good as you want to be. And that's okay too, right? I think there's okay with making progress, entering to new domains, and just helping people just relieve the pressure. Ideally, if you're any good at this thing, you're going to always feel this way forever because you're humble enough to keep learning, so you shouldn't feel so bad about it.ADRIANA: Yeah, that's true. That's a very excellent point. So let's get into the meaty bits. One of the things that I wanted to share with our audience was how you came to be on the podcast. We met at KubeCon North America in Chicago this year, and you were doing a book signing. And I came, stood in line, the long line. It was totally worth it. And I was wearing this mask that had the sticker for the podcast, Geeking Out, and you said, "Oh, what is that?" And I said, "Oh, that's my podcast."ADRIANA: And you said, "Oh, I could be on your podcast." So I am so stoked that you were able to join. And yeah, I mean, I've admired your work from afar for many years. I find your approach to Kubernetes very accessible, especially because it's such a complex subject matter. So I wanted to start off with how did you get into this field in the first place? Where did you find your calling to make things technical things, gnarly technical things so accessible to folks?KELSEY: I want to answer that question, but I want to address this advice that I give to my former self and to people that I run into all the time. And they say, how do you put in the effort to make sure good things happen to you in your career and in your life in general? And so you at that book signing with the podcast on your mask. You're advertising to the world, this is my podcast, this is what I'm doing, and you're advertising what needs to happen next. And so for someone like me, I can see that clearly. I understand in that limited interaction that there is this opportunity that I could actually be on your podcast, because now I know you have one. I think a lot of people really confuse luck and that kind of effort, right? When you put that kind of effort forward, you tend to make things happen. And so I just want to highlight that part of you having that as part of your strategy of going to KubeCon, making the best use of your time and every human interaction. So kudos to you for doing that.KELSEY: But it's a perfect example of how people kind of design their own careers and create the world that they want. So that's perfect. Now to your question about this whole idea of explaining things simply to other people. When I was getting into tech, a lot of people come from various backgrounds I come from the...fast food was my only job background, and I didn't go to college. And so for me, learning technology was like a pivotal life-making decision. I need to get into this field. I admire people that are in this field.KELSEY: I don't know anyone that's in this field. And so I would go get all the books and just flip through them. I remember the first book I think I bought was the A+ certification guide. I was like, I'll start there. And you just go through all of this stuff and you look at all your notes, right? You're trying to simplify all the information to truly demonstrate that you understand it. And everyone knows that feeling of the A-ha! moment where you take something that is complex to you and you finally understand it, and your confidence level just goes up. It immediately goes up. And so that feeling, I've always enjoyed having that feeling because it felt so empowering.KELSEY: So whenever I had the opportunity to speak at a meetup, I've noticed that some people at meetups or conferences, they speak, and it's just like, overwhelming. Hey, here's this computer science diagram. Here's this map that you cannot understand what's happening, and they are happy with just leaving it as a mystery to everyone. And you're like, what the hell was that? You had this opportunity to let me have my light bulb moment, but you chose not to. You chose to try to overwhelm me with your vast understanding of things that I don't. And so I've tried to say, what if I can make people feel like I felt whenever I learned a new subject? So this is why I've always said, hey, now that I understand this thing, I want to show it to you as well. But before I can, I have to give you context where I came from, my understanding beforehand, and then what led me to that understanding. And then let me show it to you.KELSEY: And I try to use analogies and simple terms, and you can see the light bulb moment go off for people in the audience, and then it becomes a game changer for their own career. So for me, I think I got addicted to that. Like, hey, I don't want to talk. I don't want to write a tutorial if it doesn't have that impact on people.ADRIANA: I absolutely love that because I can completely relate to that feeling, the euphoria, the high that you get from solving a problem, especially something where you've had to really put on the detective skills hat and try extra, extra hard to solve it. So that is so wonderful. I love that so much, and I think it's so important because making learning accessible to people, I think, makes it fun too, because I agree with you. Like those gnarly architecture diagrams that just look overly complicated, and then your brain starts to wander, and then you miss some important thing, and then that's it your opportunity for learning. That thing is gone if you're watching that lecture, because it's just, like, way over your head. So I think that's so great. Such an awesome approach to really disseminating information across the industry, especially these are not easy topics to unravel, right? So, Kubernetes, for example, how did you come upon doing your work with Kubernetes?KELSEY: You know how you walk in on someone watching some hit TV show, they're on season six, right? And you ask them, what's going on? Why is this person not like this other person? They're like, I got to recap season three for you to understand what's going on on the screen right now. And so I think for a lot of people, Kubernetes was my season six, right? I had always been in tech trying to share information. If you would have caught me 15 years ago, you would have saw me at a Python conference teaching people about packaging Python applications. If you saw me maybe six years after that, I was at Puppet Labs trying to contribute to configuration management tools using Ruby. And so when I get to Kubernetes, there's a whole career behind me of trying to build similar systems without the terminology or the experience. You just know that there has to be a better way of doing things. So when I saw Kubernetes for the first time and really got hands on time with it, there was an a-ha! moment. I was like, you know what? All the scripts, tools, philosophies, techniques, it has now been serialized into this one checkpoint, and the industry has finally given it a name.KELSEY: And so when I got that feeling, you know what was next, right? It was like, hey, I can't wait to go to a meetup to show people this thing. And I think the reason why I was able to resonate with so many people is because I had that previous background of doing things manually, trying different automation tools. And so I was just so excited. Like, I think we finally found the thing we've been all trying to build, and it looks like this. And so I think a lot of people got to see that season. It was like, oh, he's the Kubernetes guy. But there's so much historical context that goes into why I was ready to have that conversation, make those contributions at that time.ADRIANA: That's basically the classic case of, like, everything you've done up until that point has led you to that moment, and now you're ready to take on that thing. Right?KELSEY: I became a better speaker than I had ever been prior. I became a better engineer than I had ever been prior. And I've gone through all of that experience, and I was able to really articulate what was important. And I think for a lot of people who have been on this DevOps journey for a decade, nothing is working. We're doing all of the things: CI/CD pipeline, infrastructure-as-code. We're missing something here. And I think the industry had overly focused on automation and not abstraction. And Kubernetes was that final thing that you could touch to say, there is a difference between automation and abstraction.KELSEY: And I think when people saw those new APIs, in many ways, I told people Kubernetes was like this type system to infrastructure. It was like a standard library that we'd never had. It's not like a thing that if you just install, it solves all your problems. But it's definitely a much better checkpoint than what people were doing before.ADRIANA: Yeah, absolutely. And it's one of those things where I feel like it's a bit of a love hate relationship with Kubernetes. Right. Because in some ways, it makes life so much easier, and then in other ways, it's like, oh, my God, this thing is so complex to try to unravel in your mind. Right.KELSEY: I want to address that a lot, because there are some people that think I am the biggest Kubernetes fan in the world, and I am not. I actually spent the last four years working on replacements. I spent so much time at Google Cloud working on serverless just to make Kubernetes go away. I learned everything about it because I think the best people that will replace it are the people who understand it the most. And the way I look at Kubernetes is very different. People look at it as a tool that is competing with their other favorite tool or some alternative ways of doing things. To me, Kubernetes is just another word in the dictionary, and my focus has always been, what does it mean? And as a contributor, what should it mean? And when I think about it as an aggregation of the previous ten year set of techniques, and you push them all together, you get this thing. And I study that thing for, like, wow, we've come a long way since those days.KELSEY: Also, you can see what's missing. And I think that part is where, for me, that's inspiring. Oh, this is what's missing. So this is where the opportunity space is. Go work there and solve that problem. But I think a lot of people get into, oh, this thing is too complex. And I always ask them, but do you understand it? If you don't understand it and you say it's complex, then I think that's a mislabeling of the situation. You can just say, I don't understand it, therefore, I don't know why I would use it.KELSEY: And I think that's a fair way to start the conversation. I think a lot of people are just dismissing it because it's complex, and I can do something much simpler, and then they tell me what they're doing. I'm like, that sounds like a Rube Goldberg machine. You just named 25 pieces of custom tooling so you can avoid using Kubernetes. I don't know if that adds up.ADRIANA: Yeah, that makes a lot of sense. I think it makes sense too, like what you said earlier about looking for something that could potentially replace Kubernetes, because I also think that we tend to get into this sort of rut where we think, well, it all ends with Kubernetes. But we all know that software has evolved so much in the last 20 years. Even everyone was talking about Heroku is this awesome thing, and now, yeah, Heroku is still in the picture, but other things have come and kind of taken our attention. So where are we moving towards then in this space?KELSEY: I think in some ways things haven't changed very much in 20 years. You write code, you build the code, and you try to do some process to get it on the server so people can use the code. About 20 years, people have been doing exactly that thing. Now, how people have gone about doing that thing, that's changed at different speeds. Some people are still writing KornShell scripts right now as we speak, deploying apps at their company, and it probably works well. Then you have some people that are still using tools like Capistrano because they want to use something that's written in their favorite programming language, in that case, Ruby. And so they just want to keep everything within their problem domain. And then you have some people who prefer platforms like Heroku, Cloud Foundry, you name it.KELSEY: I think the challenge has been is lots of people have been looking for that one solution for everything. I remember when Cloud Foundry, like the Heroku competitor that you could run yourself, it was like, look, twelve factor apps are the way to go, and you can write everything as long as you use Java and Spring Boot. You do that, you're done, you're great. And then it's like, okay, that's fine. What about my batch jobs? Where do I run those? Not there. What about my databases? Where do I run those? Not there? And then what happens is you end up having to bring in a second or third platform. And that's where the harsh reality of all of this stuff is, is that whenever we don't have one solution to solve everything, you end up having to complicate your infrastructure. And I think complicated infrastructure just the actual norm at this point.KELSEY: What the world wants in terms of if you have a public facing website, you're probably going to have a cache, you're going to have Cloudflare DDoS protection. Various security concerns that Kubernetes versus Heroku is such a small part of the decision making process that even if you got that layer right, it is such a small part of the equation that thinking that's where the complexity is, ignores the big picture, where I think things like Kubernetes are 1/100th of the equation.ADRIANA: Right. That makes a lot of sense. Now, on a similar vein of Kubernetes like product, you've also done some work with HashiCorp Nomad, right? How would you compare Kubernetes to Nomad for folks who aren't familiar with both?KELSEY: Respect to everyone that contributes. Because I've written lots of code myself, and you do the best you can. So we just got to make sure we get that out of the way. We're not attacking people here. So if you have a HashiCorp logo tattooed on your body or Kubernetes logo tattooed on your body, this is not about you at all. When I first saw Nomad, I remember, is when they announced it in Portland at one of the smaller first HashiConfs. And I was scheduled to give a talk about Kubernetes, and I changed the talk the night before to do Nomad versus Kubernetes. And I remember Mitchell, Armon and so many people from HashiCorp standing there watching the talk. Everyone's crowded in to watch the talk.KELSEY: And look, I'm not a mean person, so I'm not someone that's naively attack a project that I'm not working on. Doesn't make any sense. But I did learn it, got it installed. And the things I liked about Nomad, you got this single binary written in Golang. You just put it on the server and it's almost immediately ready to go, starting getting value, right? That part around, just go get a binary and just have it run on the server. It really, really made that easy. The part that wasn't great, though, is the API. You look at it and it's like, what is this thing? Right? I think I get it.KELSEY: And it felt like, oh, you're trying to copy the Borg white paper about what a task is, but you haven't used Borg enough to know that this is not what you want to copy. And so it was a good serialization of that knowledge that was out there. They built a very high performance fast scheduler. They optimized for scheduling, speed, and performance. But the thing I think that they missed was the ecosystem. This space now is about collaboration. So you have lots of people who want to build infrastructure, automation, tools. And the one problem we've had over time, in my opinion, is that you have to glue them all back together.KELSEY: And scripting only gets you so far when you have to glue together all these various APIs. So Kubernetes takes a different approach. Kubernetes says these things are all related. Your load balancer and your app and your IPs, and your storage, your secrets, all of it is related. And they depend on each other. And so Kubernetes felt like it lived a life where the maintainers or the people of that project had been using Borg for a decade or two and said, what would we fix? And they come into a popular ecosystem like Docker and all these pieces, and they aggregate them. And when you look at the API, you can see the experience peek through. Right here is a pod.KELSEY: A pod has to have multiple containers because most apps that people deploy in reality, need things like NGINX or sidecars or logging daemons. And so I felt like Kubernetes had so much more experience baked into it than just being a faster, easier to manage system for deploying things. So given that, it was really nice to see over time that both communities kind of learn from each other. I remember when Nomad started adding things like volumes, sidecars, or other things that you would typically see in Kubernetes. So I think some people like Nomad because of its simplicity. I kind of lean towards the simplicity side of the house, so I kind of resonate with the whole Nomad thing. But watching people kind of glue together, like vault console, and all these other pieces to try to get a whole system, I'm like, man, at this point, now Kubernetes starts to look a little better.ADRIANA: Yeah, I definitely agree. I worked at a job where so I had come from a Kubernetes background and worked at a job where it was a Hashi shop, and they're like, oh, we're using Nomad. So I'm like, oh, my God. How do I translate this? And when I learned that Nomad is not fully equal to Kubernetes, that you have to still stitch these other pieces together, I'm like, oh, okay, that complicates things. But I definitely agree with you. One of the things that I do appreciate about Nomad is that certain things seem a little bit simpler. And I did find the learning curve not too bad. Maybe it was because I also knew Kubernetes at the time, so maybe that helped and it allowed me to translate.ADRIANA: But there's definitely a lot of stuff that I appreciate about Nomad, and I'm glad that I've had exposure to both ways of doing things, because I think that's really cool. And like what you were saying, both communities learning from each other rather than, like, let's hoard our secrets, because that way you can end up with better products overall, right?KELSEY: 100%.ADRIANA: Now, one thing that I wanted to ask you about was your famous Hashinetes tutorial. What motivated you to put this together? And also, if you can just share with folks what this Hashinetes thing is.KELSEY: I remember the Hashinetes talk, because that was the year I was like, okay, all of these tools have been out for a while. Vault is out. Consul is out. Nomad is out. Kubernetes is out. Now what? How do you think about all these things? What do you even do with them? And I remember that year I wanted to have fun, right? Previous years, it's more about, what are these things? And then maybe years after that, it's like, it's in production. But I was like, you know what? I want to have a irresponsible talk. I remember starting to talk off: "Today we're going to be irresponsible."KELSEY: "Do not do this in production." "Do not go to work and say Kelsey said anything." This is just having fun. Okay, and so I remember having a Kubernetes cluster or maybe even Nomad, and said, all right, we're going to install Nomad as an app to see how it works. And I just started adding different layers and components one by one. Number one, teaching people how all of these things actually fit together and how another scheduler could actually arrange them and put them into place. And then I think people had so much fun with the talk. It's like, wow, look how powerful these tools are that they can actually deploy and manage each other if you really wanted to.KELSEY: And look how they're similar in some ways. And I think a lot of people were like, oh, these are just you need to pick one or the other. And at that time, there was a blog post of a company using Kubernetes for some stuff and then using Nomad for some of their batch jobs that would benefit from the Nomad way of doing things. I thought that was just, like, the right way to think about it. So that talk Hashinetes is like, what happens if you push Kubernetes and all the HashiCorp tools together, like using Vault for secrets instead of the thing that was built into Kubernetes, because I think Vault was a far superior secrets management product and API. And then what if you were to use Consul instead of Kubernetes built-in service-discovery? What would you get? And then let's just say you really do like Nomad. What if you were to run that inside of Nubernetes, too, and let that become the scheduler instead of Kubernetes doing the scheduling? And I think when people kind of saw that talk, they understood how to really fairly evaluate those tools. So we just had a bunch of fun.ADRIANA: What do you think was the biggest learning from putting this talk together for yourself?KELSEY: I think, honestly, if you just live 100% in Kubernetes land, all you know is config, maps, secrets, and you have an idea in your mind that there's no other way of thinking about these problems. Right? Everything must be a CRD. Kubernetes, Kubernetes, Kubernetes. But I think people forget I was a contributor to Kubernetes. I knew how some of the inner workings worked. And so it's like, how do you get Vault to work nicely inside of Kubernetes? Then you have to rethink the APIs, and you start, oh, the Kubernetes secret management API isn't that great at all? And so when you bring in Vault and you have to stitch it in and bake it into the whole process, you really do gain empathy for gluing all of these parts together yourself. So I think the biggest learning for me is that, number one, you can do it. There are situations where it does make sense.KELSEY: Think about it. If you have multiple clusters and you want to have multi cluster service discovery, you cannot do that with Kubernetes alone. When you add something like Consul, you can have Consul be the place that takes over DNS. And guess what? Voilà, you can now address multiple clusters using one service discovery tool. And so it's like, oh, okay. So even though Kubernetes hasn't solved all the problems, it doesn't mean that you can't bring in all these alternative tools to step in and fill that gap.ADRIANA: And it's nice to see that everything plays nice in that little ecosystem and that you can, I guess, take advantage of each tool superpowers, right, to sort of give that boost to Kubernetes Awesome. Now, on the Hashi front, I also wanted to talk to you briefly about a talk that you gave at KubeCon EU, "From Community to Customers". And I attended that talk, and I really enjoyed the talk. I thought it was very interesting how you were talking about this fine line of what to keep open source versus what not to keep open source. One example that you cited was HashiCorp, and then shortly thereafter, HashiCorp changed their licensing. So what are your thoughts around that?KELSEY: Yeah, I actually had this question come up a few times, and I always tell people from a place of empathy, I had a project, Confd. It became a little popular. I remember going to FOSSDEM on the other side of the world in Europe, and watching someone give a talk about using Confd, this miniature configuration management tool, and how they were using it and why they thought it was one of the greatest projects ever. Like, as a maintainer of an open source project, you'd love to see a community form around the thing you've built. But as a solo maintainer, you also know how hard it is to say no. And you wake up on, like, a Saturday morning and it's like, hey, I work at a huge company that makes tons of profit, and I get paid really well to do my job. I would like you to work for free and add this feature that we really, really need to make even more money. And you're like, no, this is not my priority.KELSEY: Number one, you're not paying me anything. And then two, you know what? You're going to have to prioritize that itself and maybe step up and do some contributions. And so when you think about it that way, and as someone who's also contributed code to HashiCorp products in the past, I did those contributions to scratch my own itch. And I understand that once I deliver those changes, it's on the HashiCorp team to maintain them forever. And so I understand the relationship here is me contributing code is not the end of the story. And so when they make that licensing change, I put myself in their shoes of trying to run a business and remember, they're a public traded company. So a lot of these decisions are not in fully their control anymore. The market wants to see profit growth.KELSEY: I don't know if you've ever worked at a profitable company, people listening to this. But having stagnant revenue year over year is a fast way to get shareholders to leave investing in your stock. So now they have this added pressure of no longer just making the open source community happy. The people that they kind of started their careers off of, now they have to try to make the market happy. And there you get into different behaviors. So now you got to figure out where to get revenue from. And if you ask someone, Where do you get revenue from something that is given away 100% for free? Last I checked, most people do not pay for things unless they have to pay for things. And so you got to draw the line somewhere.KELSEY: And I think the big controversy is, where do you draw the line? Do you draw the line on the core of the product? NGINX tried to do things like that. It didn't work out well over time, do we draw the line on, like, enterprise features and Web UIs? Right? That could be a fair place to draw the line. And so I think for a lot of people, HashiCorp decided to draw the line at commercial competition. If you take our software and start competing against us, using our name, likeness, whatever we say now in our new license, the business source license, that you can't do that. And so if you're being honest, as a user, don't really care. Like, I don't plan to start a business competing against terror. If you're being honest, I literally don't care.KELSEY: And most people don't really exercise all their open source freedoms anyway. I'm not saying that's not a good reason not to have them, but a lot of these licenses like Apache 2 to me to fully realize the benefits of them. I think you do need to become a contributor to really understand what the code base does, be willing to step up to fork a project when the time comes and having the skills to maintain it. A lot of people don't understand that's the other part of this deal. And so when they change that license, I think people got a wake up call. They own that project. It is not our project. Even those with that HashiCorp logo somewhere tattooed on their body, it's not your project.KELSEY: It belongs to HashiCorp. And so now I think there's a rethink. And a lot of people forget HashiCorp predates the CNCF, right? So they're not a part of a foundation, even though a lot of their technologies are foundational, TerraForm, Vault, those things belong to HashiCorp, a private company doing what they have to do. And so for me, I look at that business license change and says, great, they made their stake in the sand. From a business perspective, this will be good for HashiCorp. Now they can say no. And now their terms are a bit clear and no longer vague. Now, for the community that is upset,KELSEY: now it's time to exercise those open source rights we've all been talking about for so long. You get to fork the project, you get to maintain the project, bug, fixes security, fixes new features and then ask the question how compatible should you remain going forward with the thing in which you branch from? That's what's on the table. So those are my thoughts on it. It's very pragmatic. I think it's one of ownership and responsibility and no matter how you feel about it, you're going to have to take on ownership and responsibility going forward.ADRIANA: Yeah, absolutely. It makes so much sense and I think you hit on a very important thing when it comes to maintaining an open source project, which is maintaining it. It is a lot of freaking work and especially if it's something that you do on the side for funsies. You can only expect so much, especially if you're the solo maintainer. So also hats off to anyone who is a solo maintainer of an open source project or works with a very small team because it's a lot of work. It's a labor of love at that point, right?KELSEY: I want to make sure people understand. A lot of people may have an ops background. That's definitely where I come from. And people think dev is easy and there's the same stress that you have in operations, right. For example, if you replace a hard drive in a server with a bad hard drive, you worry the first couple of days like, is that RAID configuration going to actually rebuild on time and the hard drive is going to stop being slow before traffic comes. You worry about these things and this is why we started doing things like on-call. And when you are maintained of open source project, you know that anything you merge in will make its way to someone's production, someone you probably don't know and you're going to feel responsible and accountable for doing that. And so there's a lot of this added pressure of like, hey, I got to be able to say no and make the right decisions to make sure that no one is going to be negatively impacted by these projects.KELSEY: I think a lot of people forget that when we start to ask and I don't like the way this person runs this open source project, there is so much pressure that goes into it. So just know that there's humans behind these projects. There's a lot at stake. So if they say no to your new feature or they have to make a business license change or stop accepting pull requests for a while while they go tend to other matters, you just have to understand that just what comes with the territory.ADRIANA: Yeah, absolutely. There are humans behind those repos right at the end of the day. Well, we are coming up on time, but before we wrap up, I was wondering if there are any parting words of wisdom that you would like to share with our audience?KELSEY: I don't know if there's any parting words of wisdom, but I do think we're at this next cycle of new technology on its way, whether it's AI or LLMs, some people only know that stuff as chat GPT. And the question that I'm hearing a lot around is, like, is this thing going to take my job? And I always ask those folks, what is your job? And they say, "Well, for the last ten years, I've just been running scripts and automating things, and I'm like the same things for ten years in a row." I was like, "Listen, if that's how you would describe your job, then yes, you might have a problem when a new set of tooling comes around that reduces the need to do that." And that's always happened throughout tech. And I think what most people should probably think about is take these moments of insecurity and just do some self reflection and say, "Hey, my tools"...and I think we started the conversation this way. People tend to confuse automation to abstraction, and a lot of times, people get so comfortable automating the same things over and over, almost like a Westworld Loop, that they forget that we should rethink the thing that we're automating and ask ourselves if we should replace it with better abstractions. So I would say this this may be your very moment to pause for a second look at the work you do, and ask yourself, "Is it time for a new abstraction?" And if it is, I think that's the perfect opportunity to either go find a project that's attacking that problem or maybe even start your own that introduces the new abstraction based on all of that experience that you have.ADRIANA: Awesome. I really love that. Well, thank you so much, Kelsey, for geeking out with me today. Y'all don't forget to subscribe, and be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time...KELSEY: All right, everyone, don't forget to Peace Out and Geek Out.ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout. Hey, hey Geeking Out fans! We're taking a little break for the holidays, so this will be the last episode of 2023. Be sure to catch us again in January as we Geek Out with a fabulous lineup of guests.ADRIANA: See you in 2024. And Peace Out, and Geek Out. Bye!

About our guest:Jacob Aronoff (he/him/his) is a Staff Engineer at ServiceNow Cloud Observability, formerly Lightstep, the tech lead for the Telemetry Pipeline team, and an OpenTelemetry maintainer for the OpenTelemetry Operator project. He's spent his career in a variety of backend roles acting as a distributed systems engineer, an SRE and a DevOps professional. Jacob's focus is enabling customers to reliably send telemetry data with a focus on Kubernetes and OpenTelemetry.Find our guest on:LinkedInX (Twitter)MastodonFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:ElixirSwiftOpenTelemetry (OTel)OpenTelemetry OperatorPrometheusOTel CollectorOpenTelemetry Protocol (OTLP)OTel for KubernetesOTel Operator channel on CNCF SlackOTel End User Working GroupstatsdOpenTracingOpenCensusJaegerCommon Vulnerabilities and Exposures (CVE)OTel Operator Target AllocatorPrometheus Re-labelingOpen Agent Management Protocol (OpAMP)SignalFXAdditional Links:Adriana's articles on the OpenTelemetry OperatorJacob's Talk at KubeCon NA 2023Jacob on OTel Q&AJacob on OTel in PracticeJacob on the Maintainable PodcastAdriana on the Maintainable PodcastTranscript:ADRIANA: Hey, y'all. Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela. Coming to you from Toronto, Canada. And geeking out. With me today is Jacob Aronoff, who is also one of my coworkers. Welcome, Jacob.JACOB: Hello. Very happy to be here. I'm so happy that we get to do this. I feel like we talked about this in Amsterdam, and I'm so excited that we get to make it happen.ADRIANA: I know, right? Yeah. This is awesome. So as we start out, I'm going to do some lightning round questions. They are totally painless. No wrong answers. So are you ready?JACOB: I'm prepared. Let's do it.ADRIANA: Okay, cool. All right. Are you a lefty or a righty?JACOB: I am a righty. So I always thought I was supposed to be a lefty, and my parents forced me to be a righty.ADRIANA: Interesting. Soul of a lefty. iPhone or Android?JACOB: iPhone. I just got the new one. USB-C all the way.ADRIANA: I'm so jealous. I think I'm going to wait one more year because I want the iPhone...I don't like the Pro Max. It's too big. But I want the Pro.JACOB: It's way too big.ADRIANA: I want to wait until they upgrade the optical zoom to whatever the Pro Max offers.JACOB: Yeah, that makes sense.ADRIANA: Yeah. Anywho, go on. Okay. Mac, Linux, or Windows?JACOB: Mac for sure. Big Mac boy. Whole life.ADRIANA: Feel you. I feel you. Okay. Favorite programming language?JACOB: I feel like Go. I mean, I'm a huge fan of Go. It used to be Swift or Elixir. Those are my two a little bit more funky choices. I used to work in Elixir, and I really loved it. Definitely one of the most fun languages I've had the chance to do. Swift, I haven't done for a few years, but there are a lot of little Easter eggs around my socials that refer to Swift a lot.ADRIANA: That's why your social handle is get_sw1fty.JACOB: Exactly. Yeah.ADRIANA: Okay, I get it.JACOB: A lot of Easter eggs.ADRIANA: Nice.JACOB: Still, I was the first person to ever write a Datadog SDK in Swift, and it's still on their website.ADRIANA: Wow. That is awesome. Very nice. Very nice. Cool. Okay, next question. Dev or Ops?JACOB: That's a really hard one. Dev. I'm just going to say dev.ADRIANA: All right.JACOB: Ops is fun, but you're still doing Dev if you're doing Ops. You're still Deving. You're still Deving.ADRIANA: I like it. Especially modern Ops. Right? I mean, maybe not...well, even Bash scripting back in the day, right? Ops was more bashy, less like Terraforming.JACOB: Yeah. Back when Ops is mostly just like Jenkins scripting with Bash. That's still Dev. There's still a lot of Dev stuff in there, so it's always been like that. It's just new abstractions.ADRIANA: Yeah, fair enough. That's a really good point. I like it. Okay, next question. JSON or YAML?JACOB: It's just...I'm a YAML engineer. I can't deny it.ADRIANA: Yeah, I like YAML better. No disrespect to the JSON people out there, but I don't get it. YAML forces me to do indentations, but that's okay.JACOB: Yeah, that's all right.ADRIANA: Yeah, cool. Two more questions. Do you prefer to consume content through video or text?JACOB: Probably text. I love to read really long form things, especially, I don't know, I save a bunch of articles whenever I see them and they'll be like, ten minute, 20 minutes reads, and whenever I have some real free time, then I'll go through one or two of them and that is like my favorite way to consume. I probably consume more video, realistically.ADRIANA: Oh, really?JACOB: Yeah, I watch a lot of YouTube videos, like "How To" type things.ADRIANA: Yeah.JACOB: But I love to read more than I love to watch. Watching is too passive.ADRIANA: I get too yeah, I agree. I think that's what I find annoying about watching videos. Like, someone sends me a video link, I'm like, it better be like some short video. So if it's like an Instagram video or YouTube short, it's fine, but send me a five minute video, I'm like, I'm never going to watch it. Even if you tell me it's like the most wonderful thing in the world, I'm not going to watch it. I'm so sorry.JACOB: Or it's like, even if you watch it, you get so distracted by another thing. It's just like I don't know.ADRIANA: Yeah, I think the only way I can consume, quote unquote, a YouTube video is if it's audio only. So I'm like just doing chores around the house and listening to it, then it's okay, right? My brain is like it helps me focus better.JACOB: I feel that basically you're just podcasting at that point.ADRIANA: Yeah, exactly. Which I love me a good podcast.JACOB: Yeah.ADRIANA: Okay, final question. What is your superpower?JACOB: Superpower? I have a useless superpower. I can do a noise. I can make a noise that's really I can click with my tongue really loudly.ADRIANA: Okay, now you have to demonstrate.JACOB: I will, but it might disturb some people in this office. Okay.ADRIANA: Damn.JACOB: I don't know if that came through.ADRIANA: It came through okay over here.JACOB: It's really loud.JACOB: That was like a quieter one.JACOB: It's useful when it's like, I need to get someone's attention who knows that I can do that. And then I'll do the click, and then they'll be like, oh, there he is.ADRIANA: Nice. I like, that. Cool. All right, now we shall get to the meaty bits, which is sweet. Let's talk OTel.JACOB: Let's do it. I'm ready.ADRIANA: All right. Yeah. So I guess for starters you're involved as part of your so we both work at Lightstep, which I guess is now ServiceNow Cloud Observability. I guess you and I met because we both work in the OTel space, although we work in different areas of the OTel space. Why don't you tell folks what you do specifically around OTel?JACOB: Yeah, so I sort of got started with OTel two years ago when I joined the company working on the OTel Kubernetes story and what's going on there. Basically I came from a Prometheus shop that really heavily invested in Prometheus and I had sort of seen the great stuff with Prometheus and then some of the struggles with Prometheus and I came in and I was, you know, I now work on top of a metrics backend. What's the best way to get metrics there? OTel has the OTLP format and so I wanted to figure out the best way to get Prometheus metrics into the OTLP format and then into our backend, specifically in Kubernetes and what is the best way to do that. So sort of began this journey on the operator group, which is a SIG within OTel that works on a piece of OTel code that sits within your Kubernetes cluster, within your environment to make it really easy to deploy OTel Collectors and do auto instrumentation and things like that. And then the feature I was working on was to make it so that you could really easily scrape and scale metrics collection. So that was sort of my first foray into it. And then I started contributing a lot. I became a maintainer for the project and now I just sort of work on OTel Kubernetes stuff all the time. So thinking about new features, new ways to help users run their whole environment for telemetry collection in Kubernetes, that's really the focus.JACOB: How do we make that as easy as possible for people? There's definitely a lot to be done, but it's a really great group of people that I think think pretty deeply about this stuff and are very good at sharing and caring and not very what's the word? Nobody's really holding on to legos. Have you heard that phrase? Is that like a known phrase? Yeah.ADRIANA: I haven't heard that expression before, but I like it.JACOB: Everybody's happy to share. There's not really someone who's particularly unwilling to accept something. Yeah, nothing like that. It's really based on the merit of the feature, not the fact that you don't get to do it nice. It's a good group as a result.ADRIANA: I really like that and I can vouch for that too because I've bugged you with a bunch of questions around the operator when I was trying to understand it better. And I've also posed questions to the operator Slack Channel and people have just generally been really nice about answering my questions, which is awesome because I think definitely tech has, I would say. I'm sure it still exists. But you see stack overflows where people ask questions and then you get some asshole who's putting you down because you're a novice to the subject and you're just trying to understand it. I get none of that from the Otel community, which I love because then it makes me unafraid to ask questions and so it makes it easier to learn.JACOB: Yeah, and a thing that I try to make sure of, at least with our group, is for anybody who's like a new contributor. I try to go really out of my way to thank them for their contribution and make sure that they're sort of set up for success with what they're doing. Like, even today, someone was asking some questions on our GitHub about some operator features. I gave them their answers and they said, if you have more questions, reach out in our slack. Happy to follow up there. And so they followed up, asked some more questions. They asked for a feature that we didn't have. I was like, oh, if you make an issue for that, we can get that on the books.JACOB: It's not that hard. And then I was like, hey, this is actually really easy feature. If you wanted to contribute it, I can walk you through that process. I can show you an example of, like, here's an example that you can look at for someone who did something similar in the past and let me know if you have any questions. And that's what they're going to go do now. They're going to make their first contribution. So it's something that I'm really happy to see as not just with my group, but like, all the groups, people are really happy to walk you through contributions and make sure that you're supported. And if there's a feature that you want, people will actually take you seriously.JACOB: They respond to you with sincerity, not what's the other word? They respond to you with sincerity, not hostility. And so there are no questions that you could ask that I've seen where someone's going to really get angry at you for asking that question. And I think that that's, like, a really nice thing. It's good to see a humble bunch and not like, a really egotistical bunch.ADRIANA: Yeah, I completely agree. And I think that's why people keep contributing to OpenTelemetry, which is great. Now, as a follow up question related to OpenTelemetry, we had you on for the OTel End User Working Group for, well, two sessions. So first for our Q&A session and our OTel in Practice, which we host those two sessions on a monthly basis. And you had a really cool story, actually, about migrating to OTel within the context of an observability company migrating itself to OTel. And why don't you talk a little bit about that? I think it's so cool.JACOB: Yeah. So previously our company was on...before we had a metrics platform...we were on stated. Like, all of our metrics were recorded via statsd. Sometimes we would rewrite them in traces, which was pretty weird, or we would have them go through a proxy so that we could aggregate them in some way and get some information out of them. So we were previously on the statsd, and then we were also on a really old version of OpenTracing. This was before the OpenTracing and OpenCensus projects merged into OpenTelemetry. And so we were on that old OpenTracing version.JACOB: And so I took on this work to migrate us to OpenTelemetry for everything. Well, metrics and traces. Logs support is still in the works, but that's the next migration. But so I started this project for migrating our metrics to OpenTelemetry, at which point the metrics SDK was still in beta, or the metrics API was still in beta, the SDK was in alpha. And so the goal was to really help the people on the, you know, iterate on their designs, work on performance and really tighten up that spec. So I did that, and then I actually found a bug in our maybe not a bug, a performance issue in the metrics code, which was a result of us having to convert from the new OTel format for attributes into the old OpenTracing sorry, other way around to convert from the OpenTracing attributes format to the OpenTelemetry attributes format. The reason this was a problem was because we shared this implementation between our tracing and metrics, and it meant that every time we recorded a metric, we had to do this conversion on the fly. And it doesn't sound that bad on an individual basis, but when you're recording hundreds of thousands, millions of metric points, that's a lot of conversions and that type of thing can really add up totally. And after I gave some of this performance feedback to the team, I actually realized that we could do this OpenTelemetry migration for tracing as well, which would then get rid of this performance concern.JACOB: And so in the midst of the metrics migration, I took a pause and then we began the tracing migration. The tracing migration was much easier because it was a more mature format at the time. So that process was a bit smoother. There were a few weird things here and there. You can read about that, I think online somewhere that we have documented, maybe, I think there's some blog posts.ADRIANA: We have the recording from your OTel in Practice, OTel Q&A discussion as well.JACOB: Yeah, cool, thanks. But so we finished that migration, we went back to the metrics migration. We got to use that performance benefit. And the OTel people actually worked on a lot of the performance recommendations that we made. So we were able to finish the metrics migration as well. And so it was really neat because I love these types of migrations, because you're really just like, you'll see the phrase a lot, replacing the engine of a flying plane. It's like doing that in place. And that's really what it feels like sometimes when you're dealing with hundreds of thousands of data points per second, how do you replace your telemetry collection about that? That's a pretty challenging thing for any company, not just us.JACOB: But then when you're the vendor serving the metrics. It's like, who's watching the watcher? That type of thing. Really the most difficult part is just reorienting your brain to think about the environments correctly to be sure that when you're talking about environment A, you are sure that that's where the data should be and not somewhere else, right?ADRIANA: Yeah.JACOB: Because for most of these telemetry vendors, whether it's us or Datadog or New Relic, it doesn't really matter. All of them have a meta telemetry environment that's sort of the secondary place that they send the telemetry of their main environment to. So that's the thing that you're monitoring. That's what lets you do these migrations effectively as well.ADRIANA: Yeah. So here's a question because this is actually like a really cool use case, because when we talk about bringing in OpenTelemetry to an organization, if you're lucky and you're starting out your application from scratch, you have the luxury of factoring observability into your architecture, right? And so you can start instrumenting in OpenTelemetry right off the bat, hopefully, right? One can dream. But then you also have the so called brown field scenarios, right, where it's brownfield. I have zero instrumentation and then there's the brownfield of like, I have instrumentation, but it's out of date. And I think that's something or not out of date, but it's not up to date with a standard, which now like the standard being OpenTelemetry. And so those are two really interesting conversations to have because I think a lot of the organizations that are adopting OpenTelemetry probably fall into one of those two categories. And from talking to a lot of folks, it's interesting too, because you have this conversation of like, you start telling them, oh yeah, I work in OpenTelemetry. Oh yeah, OpenTracing, we use that.ADRIANA: And I'm like, no, not the same, not really. You're having to educate them on that. But folks are also like, even if you get them sold on, like, okay, OpenTelemetry is the thing you got to now talk about a strategy for bringing that into the organization. And that can be very tricky. I mean, where we're at, it was an easy sell because it's like, well.JACOB: Yeah, this is what we do, this is what we work on. We should be doing it ourselves.ADRIANA: Yeah, exactly. So that's not even the problem. But even with that easy...I'll say easy, right? Because you're not having to deal with that hurdle. You have the hurdle of like, well, I've got some existing stuff now that I have to migrate. So one thing I'm wondering is, as you mentioned, there was some old OpenTracing stuff in place. And one of the things about OpenTelemetry is that they say they're backwards compatible with OpenTracing, OpenCensus. Now, which from my understanding means that if you have that stuff in place, you don't have to gut it right away.ADRIANA: However, you probably don't want it to stay that way forever. So what do you say to folks who are in that position?JACOB: A real it's a benefit that OTel provides these bridges to these legacy formats so that you can start using OTel and then get all of that in place. The thing that I always think about whenever doing these migrations, whether it's like a service, your telemetry, it doesn't really matter. The question is, how long do you want to be in a dual state? How long do you want to be in a state where you're potentially confusing someone on call? It's like the real crux of the issue is it's like always imagine yourself on call for whatever service you're changing, and someone gets paged at, like, 3:00 A.m.. Do you really want someone to have to reason about where your telemetry is coming from or how it's getting generated? You don't you really want that to be consistent. You don't want to have to ask the question, oh, is this like an OpenTracing thing? Is this an OTel thing? In the same way that if you're migrating a service and you have legacy service and new service, if you're in the dual state for a long time and you get a page for an upstream thing that's related to both of these downstream services, it's really frustrating to have to ask the question, which of these downstream things is affecting me? Right? Yeah, it'd be much easier if it was just I look at the single downstream, and I know that's the problem. Basically, it's shaving the decision tree for.ADRIANA: This that you're doing.JACOB: And so anything that you can do to remove the amount of time that you're in that dual state, removing those branches is going to do you better in the long run. The migration path is good that you can do this. There's another path, which I also think is a great option, where the OTel Collector probably supports whatever format you have right now. I'd be surprised if it doesn't. What you could do is just send rather than installing a bridge into your code, you could just send your legacy format to the Collector and have the Collector output, and then you can change your application to use OTel in whatever time frame you want, and then just have that sent to the collector, which already accepts OTLP. Yeah, right. And so that'll help you actually verify that the migration worked. You're already getting OTLP.JACOB: You don't have to do anything with that. And then once you start sending OTLP from your application, you should see no difference in what's yeah, and that's a pretty verifiable thing. You could actually even use the file exporter on the OTel Collector to actually dump the data that you get. And then for Service A, run it with Jaeger for ten minutes, dump that data with the OTLP out, and then do Service A again, but with OTLP, dump that data for ten minutes, and then just see what it looks like, understand that you should see, like, a pretty minimal difference between those.ADRIANA: Right.JACOB: And that type of thing can give you so much confidence. And you can do that probably from your local environment without even needing to push it up. And so that's something that we didn't really consider as an option at the time. But had we thought of that, I definitely would have done it that way. It would have been a great option.ADRIANA: Yeah.JACOB: Where we could have just moved to OTel instantly and then backfill. Right. That's like a much easier path.ADRIANA: Yeah, I agree. I mean, it's a very low friction approach, especially at my old company. They were using OpenTracing in a few spots, and so the mention of moving to OTel kind of sent people in a panic. Like, we have to re-instrument. Yes, we do. But hopefully never again after. But that idea sent people in a panic, and I had the same thought as you, which was like, yeah, just pump it through the Collector. Like, you don't have to change your code right away, but with the intention of eventually changing your code.ADRIANA: Because now, correct me if I'm wrong, but if you continue on OpenTracing, you don't get to reap the benefits that you get with the whole OTel ecosystem, right? I mean, you don't end up with the traces and metrics correlation and the traces and logs correlation or any new updates to the API or SDK, right? You're kind of stuck with whatever OpenTracing was when it froze, when it was retired, basically.JACOB: Yeah. Which means if there are any CVEs, you're kind of like, out of luck. Which is a bad state to be in.ADRIANA: Totally.JACOB: It's a really bad state to be in.ADRIANA: Yeah. Awesome. Yeah, I definitely like that. Now, going back to the OTel Operator. So you said that you're doing mostly work around the metrics portion. It's the Target Allocator specifically, right?JACOB: That's exactly. Right.ADRIANA: Yeah.JACOB: Now it's a bit more than that.ADRIANA: Okay.JACOB: But back then, like, last year was basically all target allocator stuff.ADRIANA: Okay, cool.JACOB: I can explain it. So basically when we started this process, someone from AWS had designed this thing called the Target Allocator. The goal of it was that you could distribute Prometheus works in targets. Targets are things that are like IP addresses, like a pod, a node, your old EC2 instance, whatever it is. You then go and scrape that instance to generate metrics. Prometheus works where it's a single monolith and you have a list of targets and it scrapes those and stores that data. You have to do this because if you have more than one instance of Prometheus, there's no way to tell which instance should scrape which thing. And so you're just going to be duplicating those scrapes. With OTel, we have the benefit of we don't need to store those metrics because we're just handing them off to the next thing with OTLP.JACOB: So the Target Allocator's goal is to allow you to distribute those targets amongst a pool of collectors. So if you have 300 targets and you have three Collectors, the Target Allocator could say, I'm going to give each collector 100 targets evenly. Right, but you need to have 100.ADRIANA: Collectors then to send it to...is that what that means?JACOB: No, you would just have to have...sorry...if you have 300 targets and you have three Collectors, then it's 100 targets per collector and then you would just forward that to your destination. So it'd be like if your destination is Prometheus actually, which now accepts OTLP, you could have OTel do all of your scraping and then just send the data to Prometheus as your backend store, right? And that would be like a totally viable option.ADRIANA: Gotcha.JACOB: If you really wanted the ability to shard your scraping and scale how you scrape targets, that would be a pretty viable approach.ADRIANA: Right, which Prometheus doesn't support the sharding right now, right?JACOB: So Prometheus has experimental sharding support but it doesn't have the ability. So it can shard your scraping, but it can't figure out your querying effectively. So because Prometheus is also a database. If you have three instances of Prometheus that are scraping each different targets, you'll only be able to query...you'll have to query the right instance each time because it doesn't know how to do that communication...to ask for, "Who has this metric?" At least that's my understanding of it. Maybe they've changed that, but I don't think they have.ADRIANA: Cool, okay. Yeah, that's super interesting. And so this allows you to scrape the Prometheus metrics which are not I mean, basically you're scraping it from wherever your source of Prometheus metrics is, right? It can be whatever, it can be coming from your infrastructure or whatever. And then this thing basically does the sharding for you and then it'll send your metrics to a destination. The destination could be Prometheus itself or it could be any observability backend that supports metrics essentially.JACOB: Yeah, yeah, exactly. Cool. And that's the real benefit. I mean, we also open up by using the Target Allocator, we can be a little bit smarter as well. So the thing that Prometheus does, because it's all in one, is most of the targets that you get, you're just going to drop. The way that the scrape configs work is you get a target which has a bunch of metadata and then your scrape config determines whether or not you should actually get the data from that target.ADRIANA: Got it.JACOB: Even prior to making the request. And so usually you have to keep all of those in memory because you're constantly scraping them and you're constantly asking this question does the metadata match my scrape config? Does the metadata match my scrape config? And so forth. Whereas because we have the Target Allocator, we can actually just drop any targets that we know the Collector won't scrape okay in advance. So we only tell the Collector to process targets that it will end up scraping.ADRIANA: Okay, so it's like a filter.JACOB: Exactly. That's what we call it. We call it a relabel filter.ADRIANA: Okay.JACOB: So the real reason that this is really cool and why we added this in is because then we can also really evenly distribute targets to Collectors because we can say only. So if you have 300 targets, we use this strategy called consistent hashing, where you just hash each target and their metadata to assign that to a Collector ID. And so if you have, like, let's say, 500 targets, but you really are only going to end up scraping 100 of them after this filter, it would be better if you only tell the Collectors...if you only distribute the targets that you're going to end up scraping, because it's going to be more even rather than trying to fit in. It's the pigeonhole principle, right?ADRIANA: Yeah.JACOB: If you have three boxes and you have 500 targets, you might evenly distribute it at first, but eventually, when you go to scrape them, it might be uneven once you figure out what you're actually going to scrape.ADRIANA: Right. By the time the Collector is receiving them, you've already just gotten the ones that you want, and so it can give you an even distribution of those. So then there isn't an imbalance, basically.JACOB: Yeah, exactly.ADRIANA: Nice. That is super cool.JACOB: It's very clever.ADRIANA: Every day. Yeah, that's very awesome. So is the Target Allocator only part of the OTel Operator? Is that something that's available as part of the standalone collector?JACOB: So the Target Allocator is its own image. Like, it runs separate from the Collector binary. You could theoretically run it without the Operator. There are definitely some people that do that, but we don't support that as like, first class support. Reason why is that we do a lot of logic to rewrite. In order to make this work, you have to rewrite the Collector's configuration, and you also have to rewrite the Target Allocators configuration. It's just a bit of, like, data munging that we don't want users to have to do just because it's a little bit complicated. So we do it in the Operator for you.ADRIANA: Yeah.JACOB: There are people who will take what the Operator gives you, remove the Operator, and then just run it themselves.ADRIANA: Right.JACOB: And that's kind of a viable option. Yeah, but that's bespoke you'd have to do that yourself. And if you ask me a bunch of questions, I'll try to help you, but there's a certain point at which I can't help you. I don't know what you're doing.ADRIANA: That sounds like someone's idea of, like, a fun weekend project.JACOB: So we have a bunch of requests from people to enable the Target Allocator as part of the Helm chart, the raw Collector Helm chart. And I tried to do it, and it was so hard. It just proved so difficult to do. The config rewriting was so challenging because Helm isn't really a language. It gives you some go templating stuff, but at a certain point, it doesn't get you all the way there.ADRIANA: Right.JACOB: And so I wasn't able to make it work, and I eventually decided to give up because it was too much of a time.ADRIANA: Yeah, that makes sense.JACOB: Which is unfortunate because people ask for it a lot.ADRIANA: Yeah, that's interesting.JACOB: Yeah.ADRIANA: Now, obviously there's an OTel Operator because obviously a lot of people run the Collector in Kubernetes. Do you know, is it common for people to run collectors outside of Kubernetes? I mean, obviously, if you're not a Kubernetes shop, I would imagine that would be the use case. But how common is it? Do you know?JACOB: I don't know. I mean, I'm sure there are a bunch of people that do it, because I'm in my little Kubernetes world, I don't hear about it that often.ADRIANA: Yeah, fair enough. Fair enough.JACOB: I'm pretty isolated, but there are definitely people who just run Collectors as binaries on raw EC2 instances.ADRIANA: Yeah.JACOB: GCS instances. People are doing it, for sure.ADRIANA: Yeah.JACOB: I don't know. They probably have a whole different class of problems than the one.ADRIANA: I know we're coming up on time, but I wanted to ask you quickly. Well, by the time this episode comes out, I don't know if KubeCon will have passed, but all the same, but do you have anything coming up at KubeCon that you want to talk about?JACOB: I do indeed. So one of the main projects I'm doing for the Operator right now is adding support for the OpAMP protocol, which is a new part of OpenTelemetry that gives users the ability to do remote configuration management and agent configuration and Observability, sort of, with superpowers. And I'll be giving a talk with Andy Keller from ObserveIQ on OpAMP and how it's going to make your life a lot easier to manage these pools of Collectors that you have. So I am working on this project in the Operator group that will allow you to basically understand the topology of your Collectors in your Kubernetes cluster and also remotely configure them. Add in new features, push out updates, everything that basically allow your cluster's observability to be on autopilot for you.ADRIANA: Nice. Who doesn't love that? Very cool.JACOB: Stop thinking about it.ADRIANA: Is that part of Observability Day, or is that part of the KubeCon, like the main conference?JACOB: Main conference.ADRIANA: Nice. Very nice. Yeah, very cool.JACOB: I don't know how many people can fit in the room that I'm in, though. I thought they'd tell you that, but I guess they don't.ADRIANA: It'll be a surprise the day of.JACOB: It will. It'll be anywhere from five people to 500 people.ADRIANA: I'm always nervous for these types of things. I think on the KubeCon schedule, you can see people already will sign up for your talk and you start seeing people signing up to attend your talk. And if it's like a small number, you're like, oh my God. And if it's a large number, you're also like, oh my God.JACOB: Yeah, I'm very nervous. Yeah.ADRIANA: Is like a very big deal. But yeah, this is awesome. Very excited for your talk. Oh, the other thing that I wanted to mention also, I don't know if it's going to come out by the time this comes out, but I do want to promote it because you were on the Maintainable podcast, you recorded an episode recently.JACOB: I did indeed. I don't think that's out yet, but definitely something to look out for, though I have no idea when that'll be out.ADRIANA: We will find out. Yeah, I think when I recorded an episode, I want to say like, in the spring and it came out a couple of months later.JACOB: So probably there's a backlog of editing.ADRIANA: Yeah, exactly.JACOB: It's a whole process.ADRIANA: I feel you. I have a backlog of editing for this too.JACOB: Yeah, that's just how it happens.ADRIANA: Yeah, totally. But anyway, something to look forward to as well, so you all keep an eye out for that. Now, before we part ways, do you have any interesting pieces of advice, be it like in tech or OTel or whatever, or any hot takes that you wanted to share with folks?JACOB: I think the thing that I always say is just do something that you enjoy. If you're looking for a job, just like find something that work on a project that you enjoy. Find something that's weird and fun and doesn't really matter and just brings you some joy. I think that we all sort of forget that coding can be really fun and enjoyable and there's so many things out there that are so cool right now, especially. And there's so many things that I think have been forgotten just out of the consciousness. I used to do a lot of coding with SignalFX and Java to do UI building and games and stuff, and I haven't done that in so long, but I had so much fun doing that. So if you're looking for a job and you don't know how to do it, my best advice is to do a project that you find very fun and interesting and not just one that you think will play well on a résumé. Because if I'm interviewing you and you tell me about a project that you were so happy to do and really excited about, that's going to be ten times better than a project that you didn't really care about.JACOB: Yeah, just have fun is my advice.ADRIANA: Yeah, that is really great advice and I couldn't agree more. Yeah, and coding should be fun. It definitely puts me in a happy place when I'm working on an exciting project that I dream up some weird thing that I want to explore and then you learn so much and I don't know, you get a high. The programmer's high.JACOB: Exactly.ADRIANA: Totally down for that. Awesome. Cool. Well, thanks so much, Jacob, for joining today. So y'all, don't forget subscribe. Be sure to check the show notes for additional resources and to connect with us and with our guests on social media. Until next time...JACOB: Peace out and Geek out.ADRIANA: Geeking out is hosted and produced by me, Adriana Vileela. I also compose and perform the theme music on my trusty clarinet. Geeking out is also by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to Bento Me slash Geeking Out.

About our guest:Juraci Paixão Kröhling is a seasoned software engineer, a Governance Committee member for the OpenTelemetry project, and an emeritus maintainer of the Jaeger project. With a strong focus on observability and open-source development, Juraci has delivered talks at conferences such as KubeCon EU, KubeCon NA, OpenSource Summit, Devoxx Belgium, FOSDEM, and various DevOpsDays. With deep expertise in distributed tracing and observability, Juraci empowers software engineers to optimize their applications and build reliable observability pipelines. Currently working at Grafana Labs, Juraci continues to shape the future of observability tools while passionately contributing to the open-source software engineering community. Outside of work, Juraci is a proud parent of three kids and finds solace in the hobby of sleeping, albeit occasionally interrupted by the delightful chaos of parenting responsibilities.Find our guest on:LinkedInX (Twitter)GitHubFind us on:All of our social channels are on bento.me/geekingoutAll of Adriana's social channels are on bento.me/adrianamvillelaShow Links:Love ParadeDomain Specific Language (DSL)Julius Volz, PromLabs FounderJulius Volz Prometheus videoOpenTelemetry Collector SIG on CNCF Slack (special interest group)OpenTelemetry Collector Contrib on GitHubCanadian National Exhibition (CNE)OpenTelemetry Contribfest at KubeConOpenCensusOpenTracingRedHat OpenShiftOpenTelemetry CollectorReal User Monitoring (RUM)OpenTelemetry Protocol (OTLP)statsdOTel Logs Bridge APISLF4J (Java Logging)slog (Go Logging)OTel End User Working GroupOpenMetricsOutreachyAdditional Links:Dose de Telemetria - Juraci's weekly Portuguese language livestream on OTelRecap of Adriana and Ana's talk on Platform Engineering at KubeCon 2023Adriana’s Observability Day talk on the Observability of CI/CD Pipelines with co-speaker Reese LeeTranscript:ADRIANA: Hey y'all, welcome to geeking out. The podcast about all geeky aspects of software delivery DevOps, observability, reliability and everything in between. I'm your host Adriana Villela. Coming to you from Toronto, Canada and geeking out with me today is Jurassi. Welcome Judasi.JURACI: Thank you very much. And I'm surprised always when speaking English, people have a trouble speaking my name. That's not the case with you. You were perfect.ADRIANA: Oh, thank you! So, Juraci, where are you calling from today?JURACI: I'm calling from Berlin, Germany. Yeah, I'm freshly moved from Brazil back to Germany to Berlin. I'm here since 9 August, so I'm here less than a month now.ADRIANA: Very cool. I've been to Berlin once when I was working in Munich, I took a train to the Love Parade.JURACI: That's nice, that's wonderful.ADRIANA: This was back in 2000.JURACI: Okay, yeah, that's nice.ADRIANA: I'd never been to the Love Parade. I was like, wow, it is an experience. It was a fun experience. So that's my experience with Berlin. I hope someday to actually see Berlin properly.JURACI: Well, almost nothing here can top Love Parade. So you've experienced Berlin on its best.ADRIANA: Awesome. So before we get going with the main content, I have a few lightning round questions that I like to ask all my guests. So don't worry, they're painless and they're fun. So let's get started. So first question, are you left handed or right handed?JURACI: Oh, I'm left handed.ADRIANA: Me too. I'm so excited to meet left handed people! Yay. Left handed and Brazilian. Best combo ever. I'm slightly biased. Next question. IPhone or Android?JURACI: Oh, Android, that's easy. Open source? Well not so much, but yeah.ADRIANA: Cool. Next one, do you prefer Mac, Linux or Windows?JURACI: That's also easy. Linux.ADRIANA: Awesome. Hardcore. Die hard, very cool. Okay, favorite programming language?JURACI: Oh that's tough. I don't know, I'm using Go most of the time now, but Ruby still has a place in my heart. But I was a Java developer for so long, so I don't know, I mean a mix of Java, Ruby and Go now.ADRIANA: Nice. Most of my career was as a Java developer. Sixteen years. So I have a love hate relationship with Java.JURACI: I guess that's why I like Ruby so much, because Java provides you the security in so many aspects and Ruby is just like this nice language that is beautiful to read and it's fun to write. It might not be a perfect fit for everything, but it is a fresh view of the programming world for a Java programmer.ADRIANA: Yes, very true, very true. Actually I know a lot of people who love Ruby because I think they describe it as being like a very simple, very elegant language.JURACI: It is. Not only the language itself is very nice, but what you can do with that, you can build very beautiful DSLs with Ruby. And they really feel like DSLs, like domain specific languages. And when you build a DSL in Java, for instance, it still feels like Java. Right. But there are some DSLs in Ruby that you cannot tell it is Ruby. You really think that it's a new language built on purpose for that specific domain? I think that's what makes Ruby beautiful.ADRIANA: That's very cool. Next question. Prefer Dev or ops?JURACI: Dev.ADRIANA: All right.JURACI: I've been operating my own servers since like ever. And I love doing that. I ran my mail server for more than a decade now. I stopped a couple of years ago. I decided not to continue doing that for my own sanity.ADRIANA: That's a lot of work.JURACI: It is, but that side of operations is really very close to my heart. But still, I'm a developer.ADRIANA: I feel you. All right, next one. JSON or YAML?JURACI: Oh no. None?ADRIANA: That's a fair answer.JURACI: Well, if I have to pick one, then YAML, of course. But yeah, no...I don't know.ADRIANA: Yeah, I prefer YAML over JSon. JSON's too garbly for me. Too much happening. It gives me Java vibes like so many curly braces.JURACI: Yeah, and double quotes everywhere. In YAML you can just choose when to place it and when not to place it. And comments, I mean, you can place comments on YAML, you cannot with JSON.ADRIANA: Oh yeah, that's true. Score for YAML. Okay, next question is, do you prefer to consume content through video or text?JURACI: Okay, I'm still a text consumer, so books, articles. But I am on this verge of consuming more and more media in podcasts or presentations, like recorded presentations from conferences and so on. I find that. I think the best balance right now is a mix of all of them. There are some great content that has been provided in forms of tutorials and presentations at Kubecons for instance.ADRIANA: Yeah.JURACI: That you cannot find written anywhere, so you have to go and watch them at the same time. I find good old books very pleasant to read.ADRIANA: Yeah, I definitely have to agree with you. Text is my default. I love a good podcast, especially like when I'm doing, running errands, walking, doing housework. It just gives my brain something to do. But yeah, I agree with you that there are some cases now where video is the only way to consume the content. So you kind of have to just power through.JURACI: Had some, I don't know, I had some like, oh, YouTube. YouTubers. No, I refuse to do that. But then I think we are past that now. People are producing great. I mean, look at Prometheus, right? So Julius Volz is creating great videos on Prometheus and there is no one better, or there is almost no one better in the world that can talk about Prometheus. No bigger authority than Prometheus and him than Julius. And it's only in video.JURACI: Of course he does have his courses and some written content, but the videos are just great.ADRIANA: That's awesome. Good to know. Good to know. And final question in our lightning round, what is your superpower?JURACI: Oh, getting my kids in bed. I can do that better than anyone else. I just get them there and they fell asleep in a few minutes.ADRIANA: Yeah, that is very impressive. I just have the one daughter and when she was little, oh my God, the excuses, the excuses.JURACI: And I'm leveling up. I take care of two now because my youngest one is too young for me, I cannot breastfeed her. So mom still has to get her to sleep. But we are now transitioning also to no breastfeeding anymore. So in the future it's not going to be two kids, but three to get in bed.ADRIANA: So then you'll really put your superpower to work. Amazing. Well, awesome. Thanks for playing along with the lightning round questions. So now onto the good stuff. So OpenTelemetry...I want to talk to you about...because you are one of the OpenTelemetry maintainers, right? What's your specific role in the OpenTelemetry community?JURACI: I wear quite a few hats actually, but two of them are really big. And perhaps the biggest hat that I have right now in the OpenTelemetry community is on the Collector SIG. So I'm a code owner for a few components of OpenTelemetry Collector, especially around the contrib repository. So things like the tail sampling processor or the load balancing exporter, or routing processor and routing connector now and a few other things. And of course Grafana specific components like the Loki receiver and exporter. I'm also part of the Governance Committee or the Governance Board for OpenTelemetry. So I think that's my second biggest hat there. But I'm around in a making...I don't know...creating confusion in other SIGs as well.JURACI: I guess that's what I can describe.ADRIANA: Awesome. I don't know if you heard that sound. Yeah. So there is in Toronto right now, we call it like the Canadian National Exhibition. It's basically like, I don't know, like a two week fair amusement park thing. And this time of year they have like fighter jets, they do like an aerial show. And even though we're not that close to where it's at, I want to say it's about. Probably about 4 km away from where I am, but it's freaking loud.ADRIANA: And they practice around this time. And I think the air show is this weekend because Labor Day weekend. Yeah. Anyway, that's where that horrid sound came from. That was super freaking loud. So one of the things I wanted to ask in your OpenTelemetry role, so what does the governance committee do?JURACI: Right, that's a great question. In the best case scenario, we don't do anything, but we are effectively the maintainers from the CNCF's perspective. So we are the ones taking the decisions on behalf of the project, especially when it comes to official decisions. So if there are any resource requests, especially involving money, that needs to go through the CMCF, then it has to go through the Governance Committee, the Governance Board, and we wouldn't take a look at those requests and we decide, oh, it does make sense, or it is good for the project, or it is not very nice for the project to do that. So this is the very bureaucratic view of the government. So we sign the request and things like that. Practically, in practical terms, what we do is we take care of organizing our events for KubeCon, for instance. So we apply for specific places, for specific talks at the maintenance track, for instance, or the contribfest.JURACI: We applied for this KubeCon, but we also mediate conflicts. And this is, I think, the most important part of the GB, the governance board or the GC. The Governance Committee, as we usually call it, the GC. And that is whenever we see something that can negatively impact the OpenTelemetry community, it is our duty to act on it. So there was a case a year or so ago where at the beginning of the year, where there were some concerns about one specific thing. And as a GC member, we have to go and see those allegations and go and see what is behind it. And is there any concern for OpenTelemetry as a community because of that? And if there is, we have to act on. But it is our ultimate responsibility to take care of OpenTelemetry as a project and as a community.JURACI: I think that's mainly how I see the OpenTelemetry role. Of course, we also have a say, so not the say, but we have a say in the roadmap so we try to build or establish a roadmap for the project. But because the way that we structure the project, every SIG is independent, and every person collaborating in the project or with the project has the freedom to do whatever they want. We cannot just go to the Collector contributor and say, hey, Juraci, you have to work on this receiver here. That's not how it works. We have to plan ahead. Of course, as part of the GC, we have to think about it and think, when do we want to do AGA for the project as a whole? Do we want to graduate or not? What do we want for the future? And once we know that, once we have that kind of vision, project wide vision, we try to get the message out and tell other maintainers. So this is where we think the project should be going.JURACI: So can we try to make a concerted effort to get there as a project? Most of the time we fail, but I guess that it is useful nonetheless. We were able to get a focus on metrics for a few months, and then we got metrics out, and we also got a concerted effort around logs, and logs are now also out. So for the most part. So I think it does work, but it is a fair question because it's not very clear when we are out what the GC should be doing.ADRIANA: Well, it sounds like GC wears a lot of hats. There's a lot going on in the GC.JURACI: Well, we got to meet every week for about an hour, and that's pretty much all of the time that we have. So we can say that we have 1 hour of work per week. Most of the time, we discuss the whole hour. So we have quite a few things to talk about, but most of them are, I have to admit, they're kind of mundane. Right? So like a company, X wants to assess whether it makes sense to have a set of features for a specific platform, right? So then we go around and ask people, does that make sense? Does that not make sense for us?ADRIANA: So then how do you communicate, since the GC does, I guess, has a hand in the overall vision for OpenTelemetry, how do you communicate that information to the different SIGs to make sure that things move in that direction, even if you're not successful, but to at least communicate the idea, even if it doesn't necessarily get implemented, or maybe not implemented in the way that was originally envisioned?JURACI: Yeah, that's a good question. We have a diverse set of GC members. We have two people from the collector SIG. We have people that are not part of any other SIGs. We have two people who are part of the TC, the technical committee as well. We have people who are users of OpenTelemetry in ODIC. So what we do today is we use the other hats for the GC members so that information can spread around. So we have the Collector folks bringing information into the Collector SIG, but we also have GC members joining the Monday's maintainers call.JURACI: So we have a call every Monday for maintainers of OpenTelemetry. So we have a GC member joining that call and bringing the updates to that group. We also have a monthly GC plus TC call where we have a discussion between the two committees discussing things that are relevant for the project as a whole, but also making sure that information flows basically. And the TC is then responsible for ensuring that the technical direction of the project is set and followed by the individual.ADRIANA: Okay. Okay. Yeah, that makes a lot of sense. Cool. So now, taking a step back, my question is, how did you get involved in OpenTelemetry in the first place?JURACI: I like to say that I'm part of OpenTelemetry since it was not OpenTelemetry. Right. I mean, I was part of the OpenTracing group back then.ADRIANA: Oh, very cool.JURACI: I was actually part of perhaps the first KubeCon where OpenTelemetry had an appearance. And it was actually here in Berlin in 2017. I think it was KubeCon Europe was very small back then, I think not even like 1000 people.ADRIANA: Oh, my God.JURACI: Yeah. Comparing now, like with Amsterdam. It's a totally different vibe.ADRIANA: That was outrageously huge. And I think Chicago is going to be even bigger, I think.JURACI: So, yeah. I'm really looking forward to it. But back then in 2017, we had a, I think it was a tutorial. Me, Priyanka Sharma, who is now executive director of the CNCF, and Ted Young, your colleague, Ted Young. We were there doing an OpenTracing tutorial back then, and it was really fun. I mean, we had a 90 minutes session there, people trying to instrument their applications using Go and OpenTracing and facing all sorts of problems. And things were working, but barely, but it was fun. So I joined back then and things just evolved from there.JURACI: Right. So we got, in 2019, perhaps, we joined forces with OpenCensus, then we formed OpenTelemetry in perhaps a little bit later than 2019, but then here we are. And I started contributing with the Collector when we joined forces, because I thought the Collector was a really cool technology back then it was part of OpenCensus service, so it was already getting our attention at RedHat back then, we thought this is a cool piece of technology that can really free up people. Not free up, but to liberate people from vendors if they want to do so. Right. So people can start using the service to make translations and they can decide at the service, OpenCensus service. Back then they can decide where to send their data. For a company like RedHat, that made a lot of sense because RedHat was not, and is not, as far as I know, interested in a backend for telemetry.JURACI: But at the same time they were and still are, probably interested in getting telemetry data out of their OpenShift clusters or Kubernetes clusters. So it does make sense to have a support for a service of some sort, like OpenCensus service. When we had a disfusion of OpenTracing and OpenCensus, then I continued working on...well then I officially started working on the OpenTelemetry Collector. It was renamed, and that prevailed until today. I continued focusing on OpenTelemetry. And a few years ago Grafana got in touch with me and know we are highly interested in OpenTelemetry and we need someone who's already in the community to help us navigate this community and understand what's going on and bring what's new inside the company and help the company provide also support for the project in whatever way makes sense for the company.ADRIANA: Oh, that's so awesome. That's so awesome. It's nice to have your work recognized like that, where a company comes to you and they're like, hey, I like what you're doing, it's wonderful.JURACI: Yeah, I can say that I'm really blessed to work with what I really like. I really like the project OpenTelemetry. I like what I do on a daily basis. I like of course, the Collector. I like writing new components and so on and so forth. But I also enjoy the community side of it. And I think that's a huge part of what I do nowadays is building bridges. It's making connections between people and it is also empowering other people.JURACI: So helping people achieve what they want, both in terms of community and their professional goals as well. I think I'm very blessed to be where I am right now.ADRIANA: I love that so much because I think it's so important. People really underestimate the power of connections and community. And I completely agree with you. One of my favorite things is being able to connect people. And sometimes you'll have a conversation with Person A, and then you'll have a conversation with Person B, and maybe the stuff that you're talking about, it's like peripherally interesting. But then person A and person B have the thing in common and you know both of them now you can connect them. And I think it's so cool to be able to make those kinds of connections and make introductions and see the sparks fly. I think that's so amazing.JURACI: I love that. And I love seeing the results a few months later as well. And then you look back and you see, oh, something came out of that and that's really cool. So I love that as well.ADRIANA: Yeah, amazing. I just want to turn back to the OTel Collector because I'm very intrigued. I had no idea that the Collector was actually like a component of OpenTracing that got ported over to, sorry, OpenCensus that got ported over to OpenTelemetry. That's so cool. When I first got wind of the OTel Collector, immediately I decided that was like my favourite thing about OpenTelemetry. I don't know why I think it's so cool what it can do. I don't know. I love this idea where especially because so many Observability back ends, they all have their different agents, and the OTel Collector is basically the vendor agnostic agent that will do all the things and will ingest the data from different sources.ADRIANA: And then my absolute favourite is being able to send it to different simultaneous sources and to be able to see, like if you're evaluating a particular vendor, you can see how the same information ends up being rendered differently by the different vendors, and then that becomes the differentiating factor between the vendors. I think that's so cool because then the vendors don't have to compete on the data format itself. Really what's distinguishing is what do they do with the data to make it useful to you to troubleshoot. And I think that's so, so awesome.JURACI: Yeah, I think that's beautiful also for the Collector, of course. But I'm looking back the industry a decade ago, right, we see that vendors were fighting for the instrumentation. So they were saying, oh, you want the best of your services, you want the best telemetry data out of your systems, then you install my agent here. And then another vendor would just come and say, oh, you should use mine. And if you wanted both, you couldn't just use both agents. They would conflict and one would very likely have troubles with the other. They cannot run at the same time. If they did, it was not on purpose.JURACI: On any issues, any problems, you would call one vendor and they would point fingers to each other. Now the situation has changed drastically today, so today, I like to say that instrumentation is commoditized. It is not where is not where the fight is right now. Of course, there are still vendors offering their own agents and whatnot, but it's not really where the differentiation is. And it's not the collector either. Or it's not the infra that helps a data from A to B. It is really on the back end. It is really how you build a scalable back end for metrics, logs, traces, profiles, and RUM and so on and so forth.JURACI: And that's where innovation is. That's where the differentiation is right now. And I think the Collector helps people who are still on the old way of doing things, and they want to get into the new way of doing things. And it is the part that you just drop into your infra without changing anything. You just drop it there and it just works. And it will just help you achieve something today right now. So you can certainly keep using your current vendor, but you can also multiplex or send out or send the same data to another vendor. And as you said, just compare the same data visualized in different ways.JURACI: I think that's why the Collector is so it gets a lot of attention or gets our attention, right, because it is such a powerful and yet easy to implement solution that allows people to get started really quick. I mean, for instrumentation, it's nice that I can follow a tutorial on a website and learn how to instrument things and learn how to apply like Java agent or the Go auto-instrumentation and so on and so forth. But the practical results on my daily routine, they take longer to reflect than the Collector. I think that's what makes the Collector very special. And it is also the versatility of the Collector right now. I mean, just today someone was mentioning, oh, I'm playing with the tail sampling processor, and I'm applying span metrics after that. And of course, we know this is a problem, right? And the person came to that realization that it is a problem, because now I'm doing metrics only on 10% of my traces. So how do I solve that? And then with the Collector, in like five or ten minutes, I was able to get into a configuration where we have connectors and we have traces going in and connectors sending the same data to two different pipelines.JURACI: And one of them is with the span metrics, the other one with the tail sampling, and that's it. Voilà. It's like a chef of the cuisine that would just get a recipe there for a very nice and tasty dish there. So I think that's what I like about the Collector as well, its versatility.ADRIANA: Yeah, absolutely. And it's so cool to also see the different because there are so many different receivers available for the Collector now where it can ingest from so many different data formats, which is awesome. So then it's like exactly what you said. You can just drop it in. You don't have to disrupt your existing system. I think I've heard some scenarios where people were using statsd and it's like, great. You can just drop a Collector there to ingest the data from statsd until you're ready to remove that part. You can just keep it running business as usual, which is really nice.ADRIANA: And then the other thing that I find interesting and not necessarily a Collector thing, but an overall OTel thing, that when OTel started, I think there were very few vendors that used to ingest data in the OTLP format. And so there were different exporters for these different vendors. But it's been really cool to see many, many vendors now being able to support the native OTLP format, basically rendering these exporters obsolete, which is amazing, right? Because it just goes to show how many vendors are actually taking OpenTelemetry seriously.JURACI: Absolutely, yes. And it is a quite different view of the road from a few years ago. Right? I mean, a few years ago we still had vendors wondering if this OpenTelemetry thing is here to stay or not. And can I just perhaps rename my monitoring pages to Observability and be done with it? Can I ignore OpenTelemetry altogether? They realize it's not the case, so it's not enough. And they have to at least ingest or accept that OpenTelemetry exists. They don't have to be part of the community, so we don't have this requirement. They can just live on their own island. They can ingest OTLP.JURACI: That's fine. Their customers are requesting them to ingest OTLP. So that's why we are doing our customers, all of our customers, they do want to generate, know, commoditize instrumentation that we talked about before, and now they want to send data to you because you are providing a very nice solution there. And if you don't, they're not going to give you another chance. They're going to go to another vendor. And I think that's how it is today and I think it's beautiful where we are right now.ADRIANA: Yeah, I really love it. When I first started learning about OpenTelemetry, I want to say it was around 2021 and I was working at Tucows and I was running an Observability practices team there and traces had not even been in GA yet. And I'm like sitting there telling no, no, this is going to be a big thing, you just wait. And I'm so so happy to see how much it's grown since then. And now we're at a point where metrics, I believe are in GA. I think logs are stable, right? I think at this point?JURACI: The data model is. We don't have a logs API and we're not going to have one, right?ADRIANA: Right. Right.JURACI: Of course we realized, I think it was clear to everyone since the beginning, but there is an official realization that it doesn't make sense for us to come up with a new logging API. Logging is the older of all of the telemetry signals people have since they started writing code. So it doesn't make sense to try to come up with a new logging API and hope for people to use our APIs in the future. Coming from a Java road and you too, you can probably name more than five logging APIs there and logging frameworks, and we don't want that. We don't want to deal with that kind of problem at the OpenTelemetry level. So what we are doing is we have the Logs Bridge API and that is something that we can implement in every language, or almost every language, and interoperate with the instrumentation that people have today. So if they have SLF4J, for instance for Java, we can have an implementation of that for OpenTelemetry. So users still use the SLF4J API if they want and they have an OpenTelemetry implementation or the new slog library for Go.JURACI: So we can implement a handler for that. And users are just going to use slog libraries for their code. But during the initialization of the logging library we can configure that to spit out OTLP, right?ADRIANA: So basically for every, or I guess the goal at least is for every logging library out there, there's going to be like a logs bridge API that basically is that connector between taking that existing logging library and converting it to OTLP.JURACI: Of course all of the libraries out there is a little bit too much, but...ADRIANA: I guess the more popular ones I would imagine.JURACI: And I guess that goes into a nice other side discussion that should that belong to OpenTelemetry, should that kind of work belong to OpenTelemetry? Or do we expect the logging framework implementers to provide such a bridge? Right, the same for instrumentation. So do we expect database client developers to integrate directly with OpenTelemetry or do we expect the OpenTelemetry community to provide instrumentation libraries for those database drivers, database lines?ADRIANA: Yeah, that's actually a really excellent question. Has it been answered yet, or is it still under debate?JURACI: Well, it is something that we did talk about during our OpenTelemetry Leadership Summit earlier this year. And the long term vision is of course that OpenTelemetry would become so successful and so pervasive that people are going to use OpenTelemetry everywhere. And we don't need to do the instrumentation on our side. People who are domain experts and coding experts on their side, they can do the instrumentation with OpenTelemetry libraries better than we can do it. And that free us up from the burden of the burden. Burden is probably the wrong word, but the burden of creating instrumentation libraries, so.JURACI: We have to maintain them.JURACI: So we have so many instrumentation libraries out there for Java, for go, it's impossible for the limited amount of maintainers that we have to keep them all up to date across all of the versions of all of the libraries. So it's a huge amount of work, and even if things work today, we are not sure they are going to work tomorrow.ADRIANA: Yeah, that's a really excellent point. And I think it goes back to a piece of feedback that I heard in one of our OTel End User Working Group sessions from earlier this year, which is basically like, you've already got folks worrying about the API and SDK for each language, right? And that sucks a fair amount of time. But then now also having to deal with the third party libraries and not necessarily being an expert in that library, and you're having to rely on the goodwill of the community in some cases to be able to instrument those libraries, which is awesome that that sort of thing exists, but that is a massive, massive undertaking.JURACI: Yeah, it is something that we have to think about for the future. We can start thinking about that today. And there are examples of projects doing that today, like using OpenTelemetry natively. But until Open Telemetry is like the winner or the perceived winner for all of the signals, or at least for metrics as well, then it is not going to be adopted by other projects natively. So if I'm a maintainer of a project that is starting right now, and perhaps it's becoming huge success in the future, do I really want to tie my users to this library here or to that library there? And if there are no clear winners for that right now, I should probably stay out. And it's perfectly understandable. I mean, for traces it's clear. We have OpenTracing, we have OpenCensus.JURACI: They're now both OpenTelemetry. What about metrics? So I think perhaps there is a discussion to have in the future, seeing how we progress. But at least for traces, we are there. So we can start that conversation now for traces.ADRIANA: Yeah, and that's really good. Interesting point that you made on metrics, because it's a question that I've had for a while, because OpenMetrics also exists. So then what's the relationship between OpenMetrics and OpenTelemetry? Do you have any insight into that?JURACI: Yeah. So OpenMetrics, we have the Prometheus working group as part of OpenTelemetry, and I think it fits there. So we have folks from the OpenMetrics project joining the Prometheus WG for OpenTelemetry. And our idea is that we should...so OpenTelemetry should be compatible or interoperable with OpenMetrics and Prometheus. So OpenMetrics is the format, is the exposition format for Prometheus, basically. So if we want to expose data in Prometheus format, we use an OpenMetrics, or we should use OpenMetrics specification for that. I think it is the other part of the project that is the acknowledgement that we are not alone and we're not alone there.JURACI: So we have to play with the other players, we have to be interoperable with the other solutions. We have to have Zipkin, Jaeger, OpenCensus receivers for the Collector. They're not going to get away. They're not going to go away, and we don't want them to go away. It's part of a healthy ecosystem to have multiple implementations of the same solution. Same with metrics. So we want very good support for not only Prometheus, but for other metrics solutions out there as well.ADRIANA: Yeah, right. Yeah. And that's a really great point, is acknowledging that there are people still using other protocols, other tools out there, and so being able to basically welcome them into open telemetry and playing nice in the sandbox is definitely an important message to give. Now, we are just about to wrap up, but before we do that, I did want to ask if there's anything that you're working on that you would like to promote and share with folks. Absolutely. I would love to share that here.JURACI: Yes, absolutely. So one thing that I'm particularly passionate about is our participations in the Outreachy program. So, Outreachy is an internship that allows people coming from an underrepresented background in our industry in it. It allows them to get in a paid internship to work on open source projects. And since 2017, back with Jaeger and OpenTracing. I try to be part of this project. And this week we got a confirmation from the CNCF that we can have one intern working. So people who are in the industry, and if you know anyone who's having trouble getting into our industry because they are part of an underrepresented group of people, spread the word and help me find those people.JURACI: And we are being part of this program once again. So the internships should start in December and finish in March, and we are going to have projects related to OpenTelemetry there.ADRIANA: Amazing. And I also want to add that you have a weekly show in Portuguese called "Dose de Telemetria" which, if you're a Portuguese speaker, definitely check it out. I'll include it in the show notes. And also to congratulate Juraci on getting a speaking spot at KubeCon, North America in...oh my God, it escapes me. Contrib fest.JURACI: Yes, that's right.ADRIANA: Yes. Congrats. And also that Jurassi is a fellow CNCF ambassador, so wanted to throw that out. Great. Well, thank you so much, Jurassi, for geeking out with me today. Y'all. Don't forget to subscribe. Be sure to check out the show notes for additional resources and to connect with us and our guests on social media.ADRIANA: Until next time, peace out and geek out. Geeking out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento me slash geeking out.

About our guest:Abby (she/her) is a Principal Engineer at Syntasso delivering Kratix, an open-source cloud-native framework for building internal platforms on Kubernetes. Her keen interest in supporting internal development comes from over a decade of experience in consulting and product delivery roles across platform, site reliability, and quality engineering.Abby is an international keynote speaker and is co-host of the #CoffeeOps London meetup. Outside of work, Abby spoils her pup Zino and enjoys playing team sports.Find our guest on:LinkedInX (Twitter)HachydermFind us on:All of our social channels are on bento.me/geekingoutAdriana’s X (Twitter)Adriana’s MastodonAdriana’s LinkedInAdriana’s InstagramAdriana’s BlueskyShow Links:KratixSyntassoVClusterHerokuOpenTelemetry CollectorOpenTelemetry OperatorHUSTEFCivo NavigateInstruqtAdditional Links:Abby at DevOps Days London 2023Abby's Keynote Workshop at HUSTEFAdriana’s KubeCon talk on Platform Engineering with co-speaker Ana Margarita Medina (sched.com)Adriana’s Observability Day talk on the Observability of CI/CD Pipelines with co-speaker Reese LeeTranscript:ADRIANA: Hey, y'all, welcome to Geeking Out. The podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And geeking out with me today is Abby Bangser. Welcome, Abby.ABBY: Hello. Thank you for having me. Super excited to talk about all of those subjects, actually.ADRIANA: Yay. I'm so excited to have you on. We had you for on call me maybe, and that was like a real treat. So I'm really happy that you're able to come on to geeking out. So, Abby, where are you calling from today?ABBY: I am calling in from London. Despite this accent. London, UK.ADRIANA: It's so cool to see where people are calling in from because there's always an assumption that it's like a North American focused podcast or probably American focused. And here we are. Super international.ABBY: Yeah. I've learned from living abroad that if people are speaking with the English dictionary, I'm just super content that I can understand what they're saying. And I actually find myself completely missing accents all the time where someone will say, oh, the Scottish person. I'll be like, oh, shoot. I actually don't remember what accent they had or South African or wherever. And it's just. I'm just really happy when I can speak the right language with people.ADRIANA: I can definitely relate. So are you ready for our lightning round questions?ABBY: I think I have to be. Let's go.ADRIANA: I promise they will be painless and fun. Okay, first question. Are you a lefty or a righty?ABBY: Righty.ADRIANA: Okay. IPhone or Android?ABBY: Absolutely, Android. All day.ADRIANA: All right. Mac, Linux, or Windows?ABBY: I have all of them. I've been on Mac most recently with software development stuff.ADRIANA: But which one do you like better?ABBY: It depends on how fiddly I am being. So I did the Linux desktop thing for a few years, and it is fun to get into all the configurations and setups, but sometimes you just want to print something. So in that world, I'm a big fan of the Mac.ADRIANA: Yeah, I feel you. I went through a Linux desktop phase and I was like, this is awesome. And then I'm like, oh, shit, I can't do anything. And this was back, I want to say, in 2007, where it was the.ABBY: Year of the Linux desktop. Just like every year is the year of the Linux desktop.ADRIANA: For me, the deal breaker was like, when I had a BlackBerry at the time and I couldn't use the BlackBerry app for syncing, I'm like, fuck. So then I ended up having. I had a Windows VM for a while. I ended up doing a Windows dual boot and then I moved to a Mac and not looked back since.ABBY: Yes, I understand that.ADRIANA: Okay, next question. Favorite programming language.ABBY: Oh, favorite programming language. So I'm working in Golang right now. I've always been someone who looks at code as a problem solving thing. So I haven't really gotten too esoteric about the most perfect language I've written in. So yeah, I just like to work. I guess this comes from my roots as a consultant of that I've written C sharp, Java, JavaScript, Python, Ruby, and then lots of infrastructure as code languages. And so I just sort of want to solve problems in the languages that people are working in.ADRIANA: I love it. Yeah, very nice. Cool. Okay, next one. Dev or Ops?ABBY: Got to be Ops. I guess if you make me choose. I love the impact you can have in Ops. Like just the scale of impact within an organization that you have when you have a really nice operations to the process. So I've been on platform engineering teams for, I don't know, the last eight or nine years, so it's hard not to pick.ADRIANA: I guess. Last week I recorded with Tim Banks and I asked him the same question. He's like, well, Ops runs the world.ABBY: I suspect we'll get into what is the true difference between Devon Ops at some point later today. But if you make me have to pick, I'll pick my roots in Ops.ADRIANA: All right, fair enough. Next one. JSON or YAML?ABBY: I really like comments so I think I have to pick YAML, but oh boy, those spaces. That's a tough one. I'll go with YAML for the comments.ADRIANA: Yes, I do appreciate the comments. Until someone pointed out that you can't do comments in JSON, which I knew but consciously was like, I think I've gotten angry at it before for not being able to do comments, but I'm like, oh my God. Yes. Two more questions. Do you prefer to consume content through video or text?ABBY: Ooh, I'll go with video. But I will say that what I do is I keep a folder of different content on my phone of what I want to bookmarks or whatever to watch or to listen to or to read. And I just use them at different times. So I separate them not by content topic but instead by way of consuming. So if I'm going for a bicycle ride, it might be auditory something, and if I'm sitting on the train it might be reading. And so I do like a bit of both.ADRIANA: But yeah, fair enough. Yeah, that's a really good point. Yeah. I was telling someone today, I love the audio stuff for when I'm doing busy work chores around the house or what you might call it, like going for a walk or for a run. Especially when I'm running. I hate and love running and it makes the time pass faster.ABBY: I can answer that one much faster than all these. Hate, hate running. That'll be the quickest of all these lightning round questions.ADRIANA: Final question is, what is your superpower?ABBY: What is my superpower? I'm going to go with being able to energize other people if I'm going for a real one. I'm coming off the back of a very exciting tag rugby win last night for a team, and I unfortunately have a bad back and couldn't play. But the team just rallies in a way that I feel like I can help bring energy to the team. And so I did my part despite not being on the pitch. Yeah, I think that's probably it.ADRIANA: That is a good talent and very translatable into the work world as well, right?ABBY: Yeah, I think it works out okay. I mean, I'm sure there's times when it gets annoying. I'm very sure that's what all superpowers, right? They have their good and their evil. But I like to think it's more good than.ADRIANA: When I remember when someone mentioned, oh, check out Abby's content. And I started watching your videos and reading your blog posts, and I remember you just came off with such infectious enthusiasm and just like, oh, this is somebody I'd like to chill with.ABBY: And then we got to. So it was great.ADRIANA: We get to see each other at KubeCon in November again, which I'm super excited for.ABBY: So excited for a trip back to Chicago.ADRIANA: Yeah, yeah. I really like Chicago, and I never spend long enough in Chicago to actually see it properly. It's the curse.ABBY: Yep. The curse of traveling for conferences is that you're always like, oh, I'm going to this great city and I will see the convention center.ADRIANA: If I'm lucky, I'll get to see the bean again.ABBY: Yeah, exactly.ADRIANA: Thanks for answering the lightning questions. So onto our regular business. So one of the things that I wanted to talk to you about is platform engineering. It is the hot topic these days. So I think, for starters, why don't you explain to folks in your view, what platform engineering is and how it differentiates from DevOps and SRE, for example.ABBY: Yeah. So platform engineering is, in my opinion, and from kind of taking together a few different quotes from other things, it's a way of building internal services to support kind of the business use cases around an organization. I think that right now there's a lot of focus on how that impacts things like developer experience for software engineers. But I'm very specifically, I feel like the definition is a bit broader than that. It's about supporting the delivery of business value at scale with consistency and security and compliance in mind. And platform engineering is the act of putting that together into services that are consumable by the organization for those end goals. I think I have a talk at DevOps Days London, where I do just boil this down to it's internal services teams done better. So it's internal, it done with more kind of product mindset in a lot of ways.ABBY: And I think where you start to differentiate between some of the other kind of buzwords and titles and philosophies that are out there, depending on what they are, is the way I've been looking at platform engineering recently is that it's an implementation of the DevOps intention. So with DevOps intention, it's to reduce the friction between silos. You looked at silos of software development and software operations and you wanted to improve the friction between that and the quickest way to do that. The best way to our understanding, was to remove the silos, ask software developers to understand their operations, ask them to figure out how to do on call and telemetry and all of these things, and simultaneously ask operations engineers to learn about software development and to bring automation to their processes and testing and all those kind of things. But that's hard. That is really hard because you're prioritizing the removing of friction over the depth of knowledge for certain people. And that sort of specialization value that comes from someone who deeply understands an area of the business or of the technology. And platform engineering, in my opinion, is also really hard just to say, but the idea is to productize those specialties and make it so that the silos can still exist.ABBY: But you're focusing on the friction between them that's being reduced. So you still have your specialists, but they're providing a service to end users that they think about like a product, and they provide in a way that is easy to access and easy to use.ADRIANA: Yeah, that sounds awesome. That makes a lot of sense. Now, follow up question. In your opinion, do you think that we've achieved what DevOps promised?ABBY: So DevOps basically promised the idea of reducing friction on the software delivery kind of process or lifecycle. I think we are achieving that. I think we have less friction than when DevOps was born 1015 years ago, twelve years ago, whatever. Do I think that we are going to get more return for the experiences and processes we've put into place over the last ten years? This is where I think we're seeing a shift in focus. So the intentions of DevOps, we are doing better, we are moving our way, but if we keep kind of trying to drill into this idea of everybody builds and runs their full stack, we're going to lose the fact that, no, we don't. First of all, we don't plug in our own hardware. Most people, most people aren't both wiring up a server and writing the CSS for their front end. Their full stack is whatever their team or their company defines.ABBY: And I think if we continue to think about the friction between the layers of value that we're providing and we keep trying to reduce that friction, we'll keep gaining benefits on this journey of DevOps. I don't know if there's an end state for DevOps.ADRIANA: And that makes sense because one of the things around DevOps is like this idea of continuous improvement and there's never going to be like perfection, right? We always chase it, but we aspire to it. Software is never done.ABBY: No. And that's what we're doing with platform engineering, bringing development to operations. We're making more software and we know software is never done. It always needs upgrades, it always needs improvements, it always needs coming back to, to make sure we're not customizing things that are now commodities. Right? So there's lots of things that you may have written a few years ago that now there's public packages to lean on and so you don't want to be maintaining that yourself. And if you don't do that, you very quickly get snowed under with the amount of maintenance you have on things that are not actually differentiating for you, your team, your business, your organization and all that. We're just multiplying software everywhere. And so we absolutely need the DevOps principles of reducing friction between teams and building and running the software that you are owning so that you have that feedback loop from production into your development lifecycle.ABBY: I think we need that as much as ever.ADRIANA: Yeah, I totally agree. And you bring on a good point in terms of like you may have developed this on your own in house at some point in time, but if there's something better that came along that solves your problem and then some, it ends up being way more awesome. But I feel like, and I fall into this trap before as well as a developer getting into this idea of like, oh my God, I get to build something cool and new and this is awesome. And then you build it and then you're like, shit, I have to maintain this. And then the shiny and new thing kind of just wears off and you're like, never mind.ABBY: Well, this is the problem with the average tenure of employees being like two years, right? By the time the average cycle is you come in excited about your new job, you spend then maybe a few months getting up to speed with how things work and what's going on, and then you've got maybe about six months to a year of whinging about something that you hate about the way that things are done wherever you are and advocating that something change. And then you implement that change. And then you move companies or you move teams and it's like, well, there's somebody doing the things that you're complaining about as you go to these new companies. And I would hearken to guess that the people who come in after you have plenty of complaints. And it's just that learning about what creates that maintainable and kind of experience that you want to have as maintaining software is hard to learn if you don't spend years and years maintaining the same software. And yet that's just not how our industry right now happens for many people.ADRIANA: Yeah, it's so true. We get bored easily or we're chasing the next shiny thing. Ooh, I get solved this problem. That sounds way cooler.ABBY: Yeah, exactly.ADRIANA: I guess the good news though is that there are companies out there that are working to alleviate some of the problems that developers and platform engineers and Ops folks are experiencing. And we've seen that with the umpteen different tools built to support Kubernetes, various platform engineering tools and whatnot. So it's been kind of interesting to see how much the market has evolved in the last 20 years.ABBY: Yeah, it is amazing. And I think you say 20 years and that's absolutely true. But there's been even more explosion in the last maybe handful of years, five years or so. As you say, platform engineering is everywhere. And the reason it is is because that feeling or that interest in developer experience has grown tremendously over the last kind of four to five years. And platform engineering is one way to possibly address this application software development experience. And so yeah, I think you're seeing all these tools that are building off of the PaaS experience, the platform as a service experience that was, I think, first really popularized by Heroku in kind of the 2012 ish era, but now that's just commonplace that there's lots of tools out there that act in that kind of way.ADRIANA: Yeah, so true. And on the platform engineering front, I wanted to ask, what do you think are some of the biggest problems that platform engineering is trying to solve right now?ABBY: So if you listen to marketing, it's all developer experience, all day, every day. It is. How can the application software developers find what their applications are doing? They need a portal and they need to make their lives as perfect and glorious as possible. But I don't actually think that is the case. I think that is a piece of things that we're trying to solve with platform engineering today. But when you talk to the companies that, I don't know, make money in things, the big organizations, they're actually not always focused on developer experience as a top reason for platform engineering. Often they have issues like compliance and security and regulation, standardization and support as a key reason. Often they have cost optimization as a key reason.ABBY: There's all sorts of reasons why providing a centralized offering as a service can benefit the business and is only auxiliary, supporting the app dev experience. But that isn't the reason for the work being done. And yeah, I don't think that gets enough press, right, because it's not as much fun as making a developer portal and things.ADRIANA: That's so true. Yeah. I mean, every time we talk platform engineering, I feel like developer experience is almost synonymous with it. And then all this other stuff takes a backseat. But compliance standardization, so important, especially, obviously in any size, but especially the big corporations that have to worry about regulators and compliance and all that scary stuff, right? I mean this is serious business. So being able to cater to that, cater to security concerns, having a standardized way to ensure that things are locked down to your company specifications, so important. And as you said, not talked about enough.ABBY: You've tripped right over one of my biggest pet peeves as well, where you mentioned that developer experience is like synonymous now with platform engineering in some ways. And also developer experience at this point is synonymous with a software application developer's experience. So someone who is writing a web app or a backend app, API app, something that is like Java or whatever, running in a container or on WASM or wherever, but the software side, they aren't the only developers. As I said in the Lightning round, I pick ops like I'm a developer, even though I write HCL code, even though I write bash, even though I write whatever chef recipes and whatever else on the back end. So I think the fact that developer experience has then I would say narrowed to the point where it's only application developers is showing, I think, in that explosion in kind of tooling and activity in the industry, because you look at all these kind of products that are coming out and they're all focused on how to get a piece of software out in front of an end user as fast as possible. But when you start to dig into how do I operate this thing, how do I build in my PCI compliance and my workflows and my marketing workflows and my customer success workflows and all these other things I need around the business, it all falls apart. And I think that too many of these forget that there's a whole other side of software delivery that isn't just writing software code. And that experience needs just as much kind of improvement, if not more.ABBY: Because again, the leverage of that across all your software teams is huge. So yeah, it's one of my little pet peeves. You chipped right over it.ADRIANA: Moral of the story, get some TLC. Outside of just developer experience, just app.ABBY: Dev experience still can be developers even, but yeah, even that's probably too narrow. I agree.ADRIANA: Yeah, very good point. Yeah, and you make a very excellent point also. You're a developer regardless of whether or not you're developing the application software or you're developing code to operate your system, it might be slightly different as to some of the things that you're concerned with, but at the end of the day, code is code. Technical debt still exists in both realms.ABBY: And the tools you have impacting what you can create exists in both realms. I think I've been on the software side for many years as well. And on the software side, I've been in a situation where an internal team provided an API that my team needed to use to create a visual front end for. And the shape of the data that came back from that API absolutely influenced and even kind of forced our hands sometimes in just what we could create on that user interface experience. Because not being able to receive the data in certain ways or access it on certain pages or whatever, I mean, the same thing goes from the platform engineering standpoint. The tools we have, the baseline kind of bash and Goling and Ruby and whatever that we're using are forcing our hands to create what I think is actually not create developer experiences. We're exposing application developers to helm charts and terraform modules and chef recipes. And it's like they shouldn't have to know what languages we write, they shouldn't have to know what tools we rely on.ABBY: But because they're all so primitive and they're all so low level. We're working so hard to piece these bits together for our business process that that's like as far as we can get on the platform engineering side or it feels that way. And so I think getting more, as you say, TLC to those platform engineers is only going to make better experiences for the application developers because they'll start to have a bit more bandwidth to be like, hold on, how are people actually using the stuff that we create? Is this the experience we want to give them? Is it about cloning a git repository or forking a git repository or something? Or should it actually be a different experience altogether? Something as a service behind an API or something else?ADRIANA: So here's a question. How do you ensure that you bridge that gap to raise the awareness so that it's not just this developer experience, application developer experience centric view of platform engineering? How do we shift the conversation?ABBY: I mean, we just need platform engineers to speak more about what they need and sort of get involved in the community. So one of the things I'm doing is I'm a part of the CNCF or Cloud Native Computing Foundation working group model. So the way it works is that there are groups with different focuses and they sort of narrow as you get down the model. And I'm in one of the kind of most narrow things, which is called a working group for platforms, and this is under the umbrella of app Delivery technical advisory groups, or tAg. And the idea is you want to be able to deliver applications. And one of the ways in which an organization can support app delivery is through the creation of a high value platform internally. And this working group is having all these conversations about identifying ways to speak to business leaders, CIOs, CTOs, CEOs about the value of investing in platform engineering experience and tooling to be able to support this kind of outcomes for them. So, yeah, get involved in the community, start kind of helping shape the conversation a bit more.ABBY: Right. There's, I think, less platform engineering voice than there is application developer voice right now.ADRIANA: Yeah, absolutely. And one of the cool things about the working group that you're part of too is it's representation from different platform engineering vendors or I guess practitioners. So you get different perspectives, different voices, sometimes competitors all working together, right?ABBY: Yeah, it's absolutely. I think there's a heavy number of vendors and a heavy number of kind of consulting oriented people, but there's also a heavy number of kind of end user related people as well. So people who work at these large, successful enterprise organizations and are driving these initiatives internal at those organizations come and speak with the working group about their experiences, about where they're feeling like they need more support from kind of the industry or from tooling and from vendors, as well as how they're piecing together these tools to create the experiences that they are. And we always want to welcome more people with more stories. I mean, we're working right now on a follow on from a white paper we released in the spring, which was around the platforms as a definition, as a white paper. We're now working on making that a bit more tangible, with a maturity model to talk about how you can kind of evaluate and grow your experience with platform engineering in relation to that white paper. And we've been through one round of edits already, and I think we have something like 30 or 40 collaborators already, and we're going into our final edits now, and we're only seeing new people pop up every day. So I really believe this is going to be a global vendor agnostic, business domain agnostic piece of work that will hopefully help people have these hard conversations at their organizations and get the kind of support they need to be successful with platforms.ADRIANA: Oh, that's really nice. That's really nice seeing the community come together like that. And what are some of the, I don't want to say grievances, but I'm going to say grievances that some of the practitioners come with when they share their experiences.ABBY: Yeah, I think that it's often just not knowing what tools to grab for and how to interrupt them all. So each tool, it's in its own right, is interesting and exciting to them. But right now, there's not enough of an ecosystem around platform engineering. Everyone's sort of fighting to be like the one tool that rules them all and says, if you use us, you don't need to worry about anything else. And the reality is that most of these organizations really, at any scale, from quite early on, will have a fairly diverse need, right? They'll have front end apps and back end apps, which will often lead to different languages, though not always. They will have applications that are more on the cutting edge and are their kind of exploration applications versus their more money making applications. They'll have ones that are compliance related and ones that aren't. And so there's just so much nuance and diversity, and yet people are trying to still sell this monolithic idea of the one platform that you can kind of buy and use, and it's just not being realistic based on the feedback we're getting from people.ADRIANA: It would definitely be lovely to have.ABBY: That if you can do it.ADRIANA: Yeah, that is a lot of stuff going on, a lot of different needs to cater to, which makes it very challenging.ABBY: Look, if you can start as a small company with using a purchased solution and keep everyone on the rails on that solution, you will be very happy. Right? That is absolutely the cheapest way of doing any of this stuff. But the way that I try and describe it is it's basically like saying that you can use another company's processes for your company, and as soon as you have chosen to create any processes out of band from that kind of platform delivery mechanism, all of a sudden you just start that fissure and you just start that divide of what can be done on platform and what has to be done off platform. But again, if you start early with a single kind of monolithic platform as a service and you can keep everybody bought in that that's the right way to go, then yes, that is the cheapest. It's always cheaper to lean on a vendor and lean on a commodity product that has its own set of engineers who are supporting and improving the feature set than to try and build it yourself and maintain it yourself. But yeah, again, we're just seeing that that isn't proving to be long term viable for many organizations.ADRIANA: Yeah, that makes a lot of sense. Now, what about taking the angle of even though each organization is different and has its own specific needs, we often find ourselves when we're moving from job to job that we keep solving the same problems over and over and over. And that gets kind of annoying because it's like, well, but I just spent like two years figuring this out of my old and now I have to do it all over again. And that sucks. So how can platform engineering help us with that?ABBY: Yeah, I think there's kind of two sides to that, or two things I'm seeing happen right now that might help with that. It's hard without a glass ball, right? But one is that you look at the kind of ecosystem that grew around docker containers and grew around helm charts and grew around operators and things like that, and you can see how they're solving some of those problems in a community way, in a publicly available way. And I think there will need to be some sort of an ecosystem that continues to build those business solutions that can be shared across. But where it becomes difficult is once a business has solved it, how much do they invest in being able to make that public? It's the same idea as when you really want to make that kind of utility. You created open source and your company is like, but now you have to scrub the commit history, you have to do all these, create a contributor guide, you have to do all these things. That's true for all these solutions as well. So it's not always cut and dry. But I think that we are building ourselves bigger and bigger abstractions and maybe we will get ourselves to a point where we can provide those higher level kind of business cases out.ABBY: The other side that I'm looking at is that we've all been here with all of the things that end up becoming commodities, like the number of times we tried to recreate the wheel on creating a user facing developer portal before Backstage came around and gave us a framework for doing that. The number of times we recreated the wheel with creating web applications that could receive HTTP requests and speak to databases before we got frameworks like rails or spring Boot or pick your language, pick your framework. We see that in order to coalesce around a common problem, there have to be a lot of attempts at solving it to really understand that it is the same. And I think you're 100% right that we're getting there with platform engineering, we're getting there with the fact that hey, we want to be able to give databases as a service, hey, we want to be able to give development environments on demand per pull request. We want to be able to update the secrets that our web app depends on as a service. There are certain capabilities that I've provided time and time again across organizations and I think we're getting there where we're going to need to do kind of more of a framework approach to things, right?ADRIANA: Yeah. And I think that's really exciting prospect, especially when you think about things like renewing certificates, updating secrets. I've heard horror stories of certificates. There's one story I heard, I don't know how true it is, but at one of the companies I used to work at, a certificate expired. The guy who was in charge of the certificates was off on a cruise. Unreachable, not good.ABBY: And it's probably one of those ones that you have to email someone and then they send you back a login to a secure portal to go download the certificate to then SCP it to the server. I've been there, thankfully not in an emergency situation, but I have been there.ADRIANA: To be able to have tooling around that where you can just alleviate that problem significantly, because nobody wants to have to keep remembering how to do this. It's annoying, it's stressful. And sadly we tend to wait until.ABBY: The last minute for it's because we have so many other fires to put out. But I mean, again, your Cert example is exactly why something like Cert Manager has become ubiquitous, right? And for any organization that hasn't moved their custom bespoke process over to something like a Cert Manager is they're being tied down and they're being weighted down by this need to maintain. Something that was very necessary before and was good that they created it. But now the best action is to move forward and start to lean on the commodities. I think I'm having a whole conversation right now with a few people about this concept of thinnest viable platform that was discussed in the team Topologies book, and the fact that right now most people are defining that really as just another word for minimum viable, as in how to get started quickly. You get started quickly with great docs, I love that. Yes, get started quickly by documenting and coalescing what you have on your platform. But call that your minimum viable.ABBY: Your thinnest viable is always having an eye towards what can you offload to a third party that you no longer need to maintain. So it's not just about starting small and growing big, but it's about starting small and staying small and figuring out how you can continue to stay at only the highest level of value that you can provide and lean on as many third parties. Open source vendor I don't care to do the things that your company is not differentiating on. Lean on the industries for that.ADRIANA: I like that a lot. It's time and money well spent. In fact, it saves you time. Maybe not money, but if you end up having to fork money over to have somebody else deal with it, because as you said, it's not part of your core competency, then so be it. Because chances are they probably do it better than you anyway.ABBY: And that's exactly because just to say money is time. I was just reading a fantastic thread about technical documentation writers, or technical writers versus software developers, and how it was somebody's personal experience where they witnessed technical writers being let go for cost savings because the software devs could write their own dOcumentation. And the thread was all about this like cool. So what you're saying is that these people who make not even half as much like the tech, the documentation people, do not make nearly half as much as the software developers. You're saying it's cheaper to let them go and take the amount of time they spend on documentation and put it onto the plate of people that cost twice as much and aren't as experienced and therefore are likely going to be slower at the job than this. So you're paying somewhere. It can be off of the books, out of your bank account if you've got the ability to do that. And if you don't, sometimes you have to pay in, we call it like sweat equity or something like you have to pay in engineering time, but you're paying somewhere.ADRIANA: Yeah, absolutely. It's funny how people don't consider, it's basically a cause and effect thing, right? It's like, oh, of course it's going to be cheaper because we've got fewer people on payroll and something that someone else can already do well, that's not normally part of their job. Now you're giving them more work, you might have to pay them overtime, or you're burning them out, both of them kind of.ABBY: Or you're getting less from them on what value you expected from them than you were hoping. Right. And that might be what, you might have made that decision quite intelligently and carefully. Or it could just be that you're learning that the hard way. The side effects of that.ADRIANA: Yeah, absolutely agree. So, switching gears a little, well, I guess related but unrelated, tell us a little bit about Kratix.ABBY: Yes, I probably should disclaimer it as talking about frameworks. So I am working for a startup called Syntasso and we're building a framework called Kratix. And the idea is that pet peeve of mine about how platform engineers just get completely ignored in the developer experience kind of conversation is what we're trying to solve at Syntasso, they absolutely won me over as one of the first tires after the founders when they talked about being in the platform engineering domain and thinking about how platform engineers can thrive, not how the end users of the platforms get benefit. And I just don't think there's enough of that conversation. And this is the business domain I'm most interested in right now. So when we think about how do platform engineers thrive, I talked a bit about what I'm seeing in trends and that is what we have decided, sort of our first product to start to create. So it's an open source framework for creating business capabilities. So you think about a platform as a service, as a kind of a monolithic thing you might be able to configure, but everything has to sort of be delivered the way that platform that you purchased or that you're using works.ABBY: Why people love that experience is because they want to have something as a service. Super easy consumable don't have to think about the maintenance behind the scenes. Someone else does that. Why they have moved away from products like Heroku and others is because they don't fit their business processes. With Kratix, the idea is that you can build anything as a service for your company relying on the tools you already rely on today, but leaning on the framework to be able to provide that sort of user interface, user interaction, that automation of the workflows, that scheduling to your different destinations, your different infrastructure, and kind of remove some of that heavy lifting from your experience as a platform engineer.ADRIANA: That's awesome. Yeah. And disclaimer, I have played with Kratix before. I wrote was a promise for installing the OTel Operator so that it sends traces to Jaeger.ABBY: Yeah, absolutely. Our first community promise provided. So first of many. So yeah, it was fantastic.ADRIANA: Yeah, that was so much fun. And it was fun because I got to provide feedback on the product on something that was so new because I think you were mentioning that whenever there were promises for other tools, you guys wrote them yourselves. And I basically sat down with the docs and bugged you all a bit here and there to write this promise. So that was a fun experience. And it was also like my first experience with the hotel operator. So I'm like, what should I do? Well, let's combine platform engineering and hotel, because that'll be fun.ABBY: Yeah, it was a great example though, right? Like that OTel Collector is itself a community provided tool that can significantly help internal organizations collect telemetry and therefore have Observability. And that's great. But where do you need this thing? You probably need it in at least more than one cluster, maybe more than one namespace within that cluster. How do you manage the deployment of that? How do you manage the upgrade process of that? Often you don't just have an OTel Collector, you have a fleet of hotel collectors, and you want them to be managed as a fleet. But our current tooling doesn't do that. Our current tooling is probably customized folders or Helm charts being installed in multiple places or something to that effect. And we really want that fleet management of OTel Collectors on demand and as a service, that's what you provided through that promise with Kratix.ADRIANA: Yeah. So that was lots of fun, and I can't wait to play around some more with Kratix. I think the next experiment which you guys helped me out with was trying to install VCluster using Kratix, which I'm like, this feels like such a fun little project to come up with. And then that was neat because it kind of exposed some stuff that I was trying to do stuff with the product that wasn't available at the time. But then it's like, okay, well now I've provided you a use case for things that people might want to try out, right? Which I thought there was lots of fun, being able to collaborate with you guys on that, just being able to be part of providing meaningful feedback to a young product that's like, it's come along quite nicely since I started playing around with it.ABBY: Yeah, thanks for that. You definitely weren't the last person to have that use case of wanting on demand clusters. And I think we talked about that platform engineering right now is pretty tunnel focused on application developer experience, but actually there's so much more to it, like cost minimization and eco impact minimization and things like that. And VCluster is a great example of where you can use that product to try and reduce the number of Kubernetes clusters that go up with all the kind of redundancy on the master nodes and all those things. And so what we did with that is be able to make on demand environments in VCluster for people. So instead of giving people a whole new cluster, you give them a VCluster. It's sandboxed, it's safe, it's less impactful on the environment, less expensive on your pocketbook. Yeah.ABBY: As I say, you weren't the last. And we're working with some design partners right now. We have a few open spots. If anyone else is in this space and is interested in working with us on the product development stuff, it's all open source, Apache 2, licensed. We're not building in private here. We're trying to build for you and for the industry. So yeah, come have a chat with us anytime.ADRIANA: That's awesome. That's awesome. Well, before we wrap up, do you mind sharing with some folks some of the things that are coming up?ABBY: Yeah, so I'm not sure exactly when we'll be airing. So some of these things might be in the past, but over the autumn I'm speaking at a few events around London, which is really fun. I love being able to not have to travel and so I'll be at a AWS day and SiVo navigate and DevOps days London. I will also will be traveling to Hungary for a Housteff, which is a Hungarian testing forum, which is a nice return to my roots. I was originally a software tester and so it's nice to be able to go and connect back with that community and then I will be in November in Chicago, as we said, for Kubecon, which is super exciting. So definitely looking forward to connecting with people there.ADRIANA: Very nice. And the thing you're doing in Hungary, that's the tutorial with TraceTest.ABBY: It is. Speaking of an amazing kind of feedback loop with a great new tool. It's not that new actually anymore, but they still are just so responsive to any questions and ideas that we have that I have. And so yeah, I'm doing a keynote about observability and have created a new tutorial around using observability with trace test for the testing audience because I think that will really kind of connect people who aren't typically in the observability space but are in the testing space with connect those two dots together. So yeah, really looking forward to that.ADRIANA: That's awesome. Yeah, I can't wait to catch the recording for that. I hope they'll be recording.ABBY: I don't know if they'll be recording the tutorial, but I am doing it on instruct so I can definitely give access to it. It's all kind of open source tech that we're going to be using. So the team at Instruqt have been so good about providing access for the community driven activities like this, and they just have such a great platform for creating content and so really makes it a pleasure to deliver.ADRIANA: That's amazing. Good to know. Thanks so much. Well, we're just wrapping up, but before we do, do you have any parting words of wisdom or hot takes to share with our audience?ABBY: Parting words of wisdom, I guess when it comes to platform engineering, just remember it's an engineering, it's software. What do we need to make great software? We need to think about how to build and run that software, and we need to think about the end users because it's a product. And so whether our end users are our colleagues or our friends and family or someone else, they're still customers. And yeah, it's a product. So I think that's really the shift in focus with platform engineering in my opinion.ADRIANA: Awesome. I love it so much. Well, thank you so much, Abby, for geeking out with me today. Y'all don't forget to subscribe. Be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time...ABBY: Peace out and geek out.ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music, Trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout

About our guest:Ori Shoshan is the Co-Founder and CTO of Otterize, where he spearheads a leading Workload Identity and Access Management platform that automates and simplifies access control for cloud-native environments. By offering a declarative and zero-trust approach to access management, Otterize empowers organizations to streamline network policy management while ensuring maximum security.Drawing from a remarkable 15-year career as a seasoned platform engineer, Ori's expertise is underpinned by his leadership roles in both technology and personnel at esteemed institutions. Notably, he made significant contributions at the IDF cybersecurity unit 8200 and served pivotal roles in cybersecurity and developer tooling startups, such as Guardicore (now part of Akamai) and Rookout (now a part of Dynatrace).Find our guest on:LinkedInX (Twitter)Find us on:All of our social channels are on bento.me/geekingoutAdriana’s X (Twitter)Adriana’s MastodonAdriana’s LinkedInAdriana’s InstagramAdriana’s BlueskyShow Links:OtterizeMicrosoft IntelliMouseBlue Screen of Death (BSOD)MacOS Kernel Panic (Mac equivalent of BSOD)Mac Spinning Beachball (or Pinwheel)Yeti (mythological creature)Python async awaitHCLINI fileMSDNOtterize Client IntentsOtterize Network MapperNetwork Mapper OpenTelemetry PROtterize CLIGraphvizOtterize CloudClay Smith (OTel PR contributor for Otterize Network Mapper)KomodorOpenTelemetry DemoKubeshopTracetestAdditional Links:Find the Otterize team at KubeCon NA 2023 in Chicago, in booth P18.Adriana’s KubeCon talk on Platform Engineering with co-speaker Ana Margarita Medina (sched.com)Adriana’s Observability Day talk on the Observability of CI/CD Pipelines with co-speaker Reese Lee (sched.com)Transcript:ADRIANA: Hey, y'all! Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada.And geeking out with me today is Ori Shoshan of Otterize. Welcome, Ori.ORI: Hi, thank you for having me.ADRIANA: Super excited to have you here today. Where are you calling in from today?ORI: So I'm actually in Berlin, but normally I am in the periphery of Tel Aviv, Israel.ADRIANA: Cool. Well, let's let's get started with our lightning round questions. Are you ready? Okay, first of all, are you left-handed or right-handed?ORI: It's a complicated answer. I know you said the lightning round was going to be easy.ADRIANA: It's all good.ORI: I'm left-handed on the computer, but I write with my right hand. You can thank my school for that. They basically made me use my right hand, so I guess I'm ambidextrous.But I tend to write...like, I basically just sign stuff with my right hand at this point in life because everything else is on the phone or typed, so yay.ADRIANA: Oh, yeah, it so you were probably born a lefty, but turned into a righty.ORI: Yeah, but I like my left-handed mouse.ADRIANA: That's interesting. I'm left-handed and I never got the left handed mouse thing, I think because the first time that someone presented a mouse to me was, like, a right-handed mouse, so I never even thought of holding it with my left.ORI: Yeah, that's a big problem. I always have to like when I buy a new mouse, sometimes I feel like as the years go on, it gets harder and harder to find, like a symmetric mouse that isn't specifically for right handed people.ADRIANA: True. That's true. Actually, you just made me think of, like do you remember? I think it was like the Microsoft mouse that was like it was shaped for right-handed people, and I remember thinking, oh, my God, if you're left-handed, you're screwed.ORI: Exactly. Yeah. So I make do with the smaller mice. But it's nice. It's an excuse to buy a gamer mice sometimes. Yeah, but it must be hard to find a cool mouse that will either be ambidextrous or tailored for left-handed mousers. Yeah, but it's a once in a few years thing. I just wish it was more like headphones, where you can basically buy the same pair again and again when they go bad.ADRIANA: Yeah, that's true.ORI: As long as they don't discontinue it.ADRIANA: Yeah.ORI: With the mice every time, I'm like, I want to buy this mouse again and they don't make it anymore, so I have to find the closest one.ADRIANA: Yeah. I feel you. I feel you. Okay, next question. iPhone or Android?ORI: Android,ADRIANA: All right. Mac, Linux or Windows?ORI: Honestly, Windows, even though I use Mac daily because I've given up because everybody else at the company wanted Macs, so I didn't want to be the odd one out, but I'm a Windows guy, through and through.ADRIANA: Interesting, interesting. I grew up on Windows, and then the first time I used a Mac, I hated it so much, and now I'm a full Mac convert. But I remember when Windows came out, I was like, this is the greatest thing ever.ORI: I think the thing that gets me is like, I've been in software development a lot of years by now and so on Windows, I really know how things work, like, beneath the outer shell, and when something doesn't work in the system, I know how to debug it. But with MacOS, my understanding only goes so far. So it's like if you see the famous beach ball spinning and something stuck on Windows, I would know how to figure out what is doing that. But on Mac, I feel useless. I'm just like, okay, better force close it and try again. Pray that it fixes itself.ADRIANA: I find that the funniest thing on Mac that ever happened to me is when I got, like, the Mac's equivalent of the blue screen of death, but it was so polite, and I'm like, oh, shoot, when this happens, I know I'm in trouble. I think I only got it once, but I was like, Damn, I should have taken, like, a picture of it. I don't even know what it looks like.ORI: I think I've had a few the last couple of weeks because I opened my laptop into, oh, everything's closed, and by the way, we've crashed. So you want to send an error report? Maybe, but I didn't get to see the actual crash screen.ADRIANA: I got to find a picture to include in the show notes. I'm sure someone's taken a picture of it.ORI: It's hard to see. They don't let you see it.ADRIANA: Yeah, that's right. It's like a Yeti sighting.ORI: Yeah, exactly.ADRIANA: Okay, next question. What's your favorite programming language?ORI: It changes along the years. Right now it's Golang definitely. I like to say it's a language for stupid people, and I'm one of those stupid people because it's really prescriptive about how you should do things. Generally, most of the time, there's only one way to do something, and I find that really valuable. I think it's insane that I can look at our own code base, like the back end and our open source code base and the Kubernetes code base, and they look pretty similar. And other languages not so much. I think Rust code bases will look very different. Don't get me started with C++, where the language basically it's worse than JavaScript. The language reinvents itself every couple of years. And there was so much sorcery to know. I probably couldn't read C++ code by now, even though like six years ago it was what I was doing all day...ADRIANA: Oh, wow.ORI: ...and before that it was Python.bPython will always have a dear place in mybheart, but it got complicated in Python free whenbthey added async await, it got kind of fragmented.ADRIANA: Yeah, fair enough, fair enough. Yeah, I do agree with you. Python...I have much love for Python. I started out in Java, and Java...I don't know, it's like coding in spaghetti. It's, like, such a verbose language, and most of my career was Java, but I'm like, nah, Java, I'm over you. I do like go. It's like, generally succinct except with error-handling. It's kind of weird.ORI: Yeah, it is weird. And it's a common refrain. I feel like a macro could help there, but I actually like the concept of like no, this is how you do things. And there are clear downsides to using Panics. You could use them theoretically as exceptions, as you would use exceptions in Python.ADRIANA: Yeah.ORI: But I like that they say this is how you do things, just do it this way. Shut up.ADRIANA: I do like, prescriptive things in that manner. I think we can do with more prescriptive languages, to be honest.ORI: It's just so opinionated. And at the same time, they leave some stuff out. Like it's not so batteries included that you don't have to do anything, which is a strange combination, but I feel it works well.ADRIANA: Yeah, definitely. All right, next question. Dev or Ops?ORI: So can I say both? I feel like probably a lot of people said it.ADRIANA: Yeah, a couple of people have said DevOps, so I'm like, yeah...but it's okay to have a preference of one over the other or, like, both. It's totally cool. Like I said, no wrong answers.ORI: Yeah, I guess both, you know, nowadays it's the popular thing to DevOps. And if you talk to people who work at, like, I think at Apple, they still separate dev and SRES a lot. And I talked to people who said they basically just throw stuff over the wall, over to Ops and they get to deploy it. And I really enjoy looking at the entire stack. So DevOps is for my spirit. I like seeing things happen end-to-end and like understanding, you know, like I said about Mac, that I can only debug so far. So I like that about DevOps, that I know what the code is and where it runs and when something strange happens, I can understand the system as a whole. That's really appealing to me.ADRIANA: Yeah, I can totally relate to that. Awesome. Okay, next question. JSON or YAML?ORI: JSON yeah, the spaces. The spaces. It's also like that spaces or tabs being part of syntax.ADRIANA: Yeah, that should be a question I could ask: spaces or tabs?ORI: Another thing I hate about Python is just there are so many discussions about that, like should we use this or that in Python too? And it's like it's meaningless and so many stupid bugs just by missing one space. No, definitely JSON even though it's annoying too. But I don't think that there is an ideal choice.ADRIANA: Yeah, fair enough. Fair enough. I did have one person say HCL, which I'm kind of down for because I feel like if JSON was, like, a little more pared down with some of the love from YAMLYeah, HCL is like a bit closer to INI files, right? INI files.ADRIANA: Yeah, true.ORI: Yeah. It is simpler to parse. I feel like I got to use them in the Windows part of my career.ADRIANA: Yeah, nice. All right, two more questions left. All right, so do you prefer to consume content through video or text?ORI: Definitely textADRIANA: That seems to be the winner so far. I've had some where it's like it depends, but it's mostly text. Word I feel like it's a little bit of a cultural thing, like in the front-end community. It seems that even JavaScript in general, I feel like videos are much more common, but growing up in the C++, no one will tell you anything. Just read man pages and like, MSDN documentation. I have a hard time when I can't scroll past stuff. So video is like the antithesis for that.ADRIANA: I know, right? Yeah, that's my biggest issue also, is, like, I need to be able to skim and search easily, and video just makes it a little bit harder. I use video out of desperation. It's like I have no other options, and all Google is giving me is a bunch of videos. I'm like, crap.ORI: It yeah, exactly. But for some things, it can be useful. And I get why it's more popular with front-end, because you want to see stuff in many cases.ADRIANA: Yeah, totally. Yeah, I definitely agree. I definitely agree.ORI: So yeah, I can understand it.ADRIANA: All right, final question. What is your superpower?ORI: Yeah. So can I say my wife? Can it be not something about me, IADRIANA: That's probably our most creative answer so far.ORI: Because I guess, like, at Otterize, we're not even two years old now. And being a founder is very demanding in terms of time and stress, and even the stress is showing up here and here [points to hairline], I like to say. So my wife has been really supportive this entire time. And just beyond not just being supportive or understanding, she talks to me about stuff, and it really helps me think things through, which can be important.And honestly, I think I would probably do a much worse job if not for her, because she really helps me process things, which is really important in such a stressful environment where you can if you're too stressed out, you can reach for knee-jerk reactions. And that's the opposite of what you want to do since you want to manage and not react. So she kind of helps center things for you.ADRIANA: That's awesome.ORI: Yeah, it's honestly, it I think that the biggest thing that has had an influence on how I do things. She gives a lot of space and support.ADRIANA: That's great. I really love to hear that. And I think this is actually a perfect segue for our topic of conversation because, as you mentioned, you are a founder of Otterize. So why don't you tell us a little bit about what Otterize is all about, how the idea came about, and what it's like to be a founder because you hear stories of the stress.ORI: Yeah, I think okay, let me start with Otterize. I was going to start with the end. So, Otterize...what we do... The name is a funny joke that alludes to what we do. So we do authorization for your back end. So it's called Otterize, because if you're Israeli and you say "otterize". Authorize and otters...they're cute animals. So it's a win-win. It's a pun.ADRIANA: Totally. Totally.ORI: So what we do is we make declarative zero-trust easy. We have released open source tooling that lets you map your Kubernetes clusters. And using that map, you can then auto-generate what we call Client Intents. So each backend service in your cluster declares its intentions, which is what it needs to access. So say I have a service that needs to access a database, an AWS IAM resource, another service. So my intent would say, I need to access this thing, and it does this in a high level Kubernetes resource called Client Intent. And then Otterize figures out how to configure your infrastructure to make it work, whether that means something like Kubernetes Network Policies or AWS IAM Policies or Kafka ACL Rules. Whatever infrastructure you have and we do that all open source with a Kubernetes operator.And I guess in one sentence it would be to make access control not a nightmare because there are so many different kinds. And as a developer, what do you want to do is I just want to call this freaking API and have it work, but then all of a sudden you've got configure IAM policies and that other service that's in a different cluster, so they have a different way that you need to authenticate and authorize.So as an aside to all that, I still get to do quite a bit of hands-on work, which is fun, and it's thanks to my awesome team that is very independent and awesome.ADRIANA: It that sounds super cool. And I think as our software is only getting more and more complex, and IAM is, I think, the bane of our existence in this space, and to be able to make it accessible, easier, less nightmarish, is awesome. And definitely very much needed in this space. And to be able to also codify in a standardized way, who'd have thought? I think that's a very awesome use case. So yay. Nice to see a tool like that in the space.ORI: It you know, as a developer, you want to say, I need to connect to that thing. You don't care if it happens with IAM and you need like a thousand different permissions, so we want to take all that away. The security team cares, you don't care, so you shouldn't care. High level.ADRIANA: Is the intent that you work like...developers work with the security team to kind of determine what those policies are supposed to be, but then the developers can, I guess, self provision that access? Is that how it works?ORI: Yeah, essentially, because what we've learned is that in most cases, security teams don't actually care to approve the policy except in very sensitive cases. Like if the ledger service for a bank has an API that transfers money so that one might be approved on a case by case basis, but in a lot of other cases, they just want to know that the access is intentional, that it's not an attacker.Just like they want to know that the code that gets deployed to production has been reviewed and approved, but they don't want to review the code themselves. And developers also don't want to talk to the security team. So both sides don't want to talk. They just want to get access and for it to be intentional.ADRIANA: Yeah. That's so great. I love that use case. Going back to we understand what Otterize does. Tell us a little bit about starting up this company. How has that been, and have you guys been around?ORI: We've been around since January 2022 and, well, it's been a crazy journey, as I guess any startup founder would say that.ADRIANA: Yeah, I would imagine.ORI: I think the biggest surprise for me would be I came into this as the CTO, and the CTO at a small size manages the dev team in the early stages. So I thought like, hey, this is going to be similar to managing a small team of developers, which I've done many times before. But what it turned out to be is I found out that when you start a new company, it's not just about managing the team because in an existing culture, an existing company, a lot of decisions have been made already.When you go to make a new decision, like, should we use this language or this library? Like the smallest decisions within an existing culture. A lot of decisions around that serve effectively as guardrails and in a new company. Even though everyone we hired were people we knew personally and worked with and have even worked together sometimes, some of them when we all joined, like day one, everybody has been to different companies. Like the previous company was different. So they all come with slightly different expectations for culture, for technical choices.So the challenge is less about how do you make...Bigger companies, the challenge is about how do you move quickly and how do you keep quality and all that. And for a new startup, I feel like the challenge is how do you end up with the right culture and do it quickly while everyone gets a chance to express themselves and feels included, while taking into account that everybody has different expectations day one, and that all happens really quickly. Like at a big company, when you start a new team, people might trickle in, but you start day one with four new people. So now you're four developers and the founders, and you're like, oh my God, so all of this has to happen at once. And everybody also really wants to work quickly.ADRIANA: Right, right.ORI: And beyond that, it just teaches you a lot about how a business works, like any business, the technical stuff, how does payroll work, all kinds of regulation. You learn a ton of stuff, which is really interesting and useful.In my day-to-day, I just got a mortgage, and I understand how banks see lawyers and people and companies and how to navigate that, which has been really useful now that I'm super busy.ADRIANA: Yeah. Um, so how did it all get started?ORI: I guess I didn't answer that. So as founders, we have experienced the pain of solving for authorization and authentication at every company repeatedly. And this is like authorization is one of those things that people just sort of accept for what it is. You start a new project or you need access to something else on IAM, so you accept that it's like this. But it doesn't have to be that way because you know, Android apps, when you build an Android app or an iOS app, you know that if you specify, "I need to open the microphone," you don't have to say which specific API it is or it's very high-level. And you know that if the user and the user sees all the permissions that they're going to need, and if they approve that, then your app is going to work.And you don't have the same level of confidence and simplicity in Cloud Native. You have to say on AWS IAM, if you're using a service that is writing to like, say, if you use Cloud Watch, it's writing to S3 buckets. And you got to make sure you also have access to write to the S3 buckets that is going to access beneath it all, which is insane. You're just a developer. You're doing one simple thing, and you have to think so deep, which, I mean, I love, but I think it slows it down if you have 500 engineers and everybody needs to understand this to make progress. And different teams use different tech stacks. We've seen people struggle with it. We've struggled with it and we see how it can be better in the mobile world. So like, why not for Cloud Native? Shouldn't accept it.ADRIANA: I think that's such a great idea. I think we see this recurring theme, too, in our space, which is, like, we keep solving the same problems over and over and over. Right? And you get to the point where, like, let me just package it into a damn tool already. Right? Because it's starting to get annoying. Like, why do we need to keep reinventing the wheel? We got better things to do.So I think that makes a lot of sense. Now, you mentioned AWS with Otterize. Does it work with other cloud providers as well?ORI: Yeah, it will.ADRIANA: Okay, cool. Nice. You got to start somewhere, right?ORI: Yeah. I think the reason no one has done this before, even though for mobile apps, it seemed kind of obvious, so both Google and Apple did it is because there are so many different kinds for other kinds of software that it's hard to even put them together. I mean, I don't think a lot of people would put network policies and AWS im policies, even though they both have the same word in their name in the same category, they would think of them as different things. It's an architectural problem, right? And you can't just imagine it all together. It's exactly the kind of problem that you would start a company for if you need to work on it for a while with a team.ADRIANA: Yeah, absolutely. Now, switching gears a bit, because I think this is the thing that got us connected initially. You all are doing some cool stuff around OpenTelemetry at Otterize, so why don't you share some of that with our audience?ORI: Yeah, sure. So we have built the authorized network mapper, which prior to OpenTelemetry, the way it worked is you set it up on your cluster, it's zero config, it's all open source and it captures DNS traffic and uses that DNS traffic to create a map of your cluster. Now, with OpenTelemetry, I've actually had a chance to work with OpenTelemetry relatively recently, but alsoa while back at my previous employer when parts of it were still called open tracing. And it can be a bit of an involved deployment to get your first few bits of data because you need to integrate SDKs and all that.So the Network mapper being an infrastructure level component that you deploy in your cluster, if it was able to export OpenTelemetry metrics, then it could make the first step to OpenTelemetry adoption a lot easier. Because I think the first step when you add instrumentation is to ask yourselves, what do I want to instrument? And if you don't know what you have, which in a large enough organization as the platform team, which might be keen on implementing OpenTelemetry, that can be where you are, that you don't even know what are the different components, which is connecting to whom, what are the dependencies that I care about.And the Network Mapper had that info and the awesome team at Lightstep/ServiceNow saw the Network Mapper and contributed an awesome pull request that exports OpenTelemetry metrics from the Network Mapper. And it was really a no-brainer for us to accept the contribution and support you guys because, you know, we think of the Network Mapper as there are a bunch of cybersecurity products with simple similar capabilities for network mapping, but we really wanted the network mapper to be a standalone thing that you could use independent of the rest of Otterize.So it's really like seeing what was possible to do with Grafana Tempo so easily with the same data that the Network Mapper has. We saw how it could make people's lives easier and really, that's what the network mapper is trying to do. To be a simple tool. You can just "helm install" on your cluster with zero configuration and get as much value as you can in a complete open source fashion.And that really works well with OpenTelemetry. So far, the Otterize Network Mapper had a CLI that was using Graphviz to create a visual map of the traffic. And you could also hook it up to Otterize Cloud to get an interactive map. But now with OpenTelemetry, you can hook it up to your Grafana instance and get an interactive map, which then helps you see, okay, I'm interested in this service and it's communicating to these three other services and that's where I want to start with OpenTelemetry.ADRIANA: Right.ORI: Yeah, it's pretty powerful...the combination, I think.ADRIANA: Yeah. And it's so cool to I think it's nice to see more and more vendors, including OpenTelemetry as part of their products, because, first of all, I think it shows the staying power of OpenTelemetry, but secondly, like, the recognition that, "Hey, this could help our customers, too." Right?ORI: Exactly. And mission number one is to make people successful in what they do.ADRIANA: Yes.ORI: And we really want Intents and Otterize and easy network mapping to be something that everybody in the Cloud Native community has access to. So we have aspirations to turn Client Intents into the way that you do authorization, even independent of Otterize, contributing to upstream.So that really falls in line with that. How do we make the network mapper more useful? Like a no brainer.ADRIANA: Yeah, absolutely. So have you seen internal and external benefits as a result of the work around the network mapper, like, with OpenTelemetry?ORI: Um, so it's still quite early.ADRIANA: Fair enough.ORI: I think it's just been out for about a week or so and we are I believe we're going to publish a blog post together that will help it get noticed.ADRIANA: Yeah, definitely.ORI: So right now, I think it's depending on people organically discovering it, which, I mean, it can happen and the Network Mapper gets organic traffic, but people who are specifically looking for OpenTelemetry and using it with Grafana Tempo are going to be best served by content that points to that. We want to add a tutorial for that too,ADRIANA: Cool. And do you have any future plans around continuing OpenTelemetry integrations?ORI: So we still need to explore that. Off the top of my mind, I'm not well versed enough to say what, but definitely I think even now there's more data that the Network Mapper has access to. Like, if you have Istio or Kafka, then it can tap into resource-level or topic-level information which can probably be reflected in OpenTelemetry as well in the same infrastructure way. And once we add more capabilities to the Network Mapper, we want to add cross-cluster discovery and the ability to discover infrastructure outside of Kubernetes from within Kubernetes. So that could also be interesting to add to the OpenTelemetry support.ADRIANA: Right. Awesome. And can you speak to how OpenTelemetry is being set up or are you guys running like a Collector as part of your internal infrastructure or...what does the landscape look like?ORI: So we're we're pretty new in terms of the internal deployment. I mean, I've had the opportunity to use OpenTelemetry before, but at other eyes we really only got into it with you guys saying, "Hey, remember Grafana Tempo?" Which we knew about before, but then we were like, "Of course." So as part of dog-fooding yeah, we started with setting up a Collector and a Grafana instance, but it's also really fresh at Otterize. We make it a point to use everything we put out, including contributions, so we know how it works for users. So we're pretty early there. But yeah...ADRIANA: Yeah, amazing how's the learning curve been for you and within the company in general for OpenTelemetry.ORI: I think there are some pretty cool tutorials and guides for we've looked particularly at Grafana Tempo which we said, videos and text.ADRIANA: Yeah, I always appreciate the tutorials with the screenshots.ORI: So Grafana Tempo has tutorials with screenshots, so getting like the initial setup has been pretty easy. But we did like I think if we built the integration completely ourselves then we would have had a slightly steeper learning curve because Clay at Lightstep did the research for us of what the metric should look like and how to configure it. So it was pretty straightforward. But I think also for a user that's trying to use it now, it would also be pretty straightforward because it was just one configuration value that we needed to pass to the network mapper after all is said and done. But I think it's interesting now to explore what else can we get from it now that we have the base layer of information there is.ADRIANA: Yeah. As a follow up, do you envision at some point adding Open Telemetry traces to your core product, for example?ORI: You mean like our back-end infrastructure or like the product itself?ADRIANA: For your application code.ORI: Yeah, because we use some level of Datadog APM to debug. So there's like a natural hook point that we would go into and hook up to OpenTelemetry now that we have it set up.ADRIANA: Right.ORI: Datadog was like we set it up just because it's the natural thing we reached for from previous experience. But we're just one step away from OpenTelemetry now. From places in OpenTelemetry.ADRIANA: You've got your foot in the door now with OpenTelemetry.ORI: Yeah, it's like all the infrastructure is there, the traceability of the code is there. Like we have contexts and everything in Go so that it's easy to pass traces. So I think we're a middleware away, which is a tiny step.ADRIANA: Awesome. That's so great. That's very exciting. And I think it's, you know, the other thing that that's super important to underscore here too is the fact that you had somebody making an outside contribution to your code base and it speaks to the power of open source, really, to be able to have your...your code's out in the wild. And someone saw this and they're like, "Hey, I got a cool use case."ORI: Yeah, absolutely. First of all, it was a pretty major contribution, which was cool. It's actually the first major contribution that we've had. We've had contributions before but they were of a bit smaller scale so it was exciting on that front as well. But yeah, the cool thing is that we were actually aware of Grafana Tempo but we didn't think to do that if we had thought about it, I think we would have built it ourselves before. But it's cool that somebody saw it, saw the potential, and just wrote the code, and now it's out there, and it's open source, and anyone can use it, and you guys contributed to it.And there's actually one other company, Komodor.io, which has incorporated the Network Mapper into their own product and their own open source, which is cool.ADRIANA: Oh, cool.ORI: It's cool to see people use it in ways we haven't thought of. That really. It's like you said, it's the power of open source that we don't need to think about every use case because someone else can think about it and add it, and all we need to do is to support and go like, yeah.ADRIANA: Yeah, totally. I mean, basically you've provided the seed and then people are just like, "Hey, what can we do with this now?"ORI: Right. And it's kind of like the pitch for Google Fiber in the United States was, we don't know what people will do with one gigabit of data, but we will build it and see what they do. So the Network Mapper is really that it's just it's raw data of your network, and it does the collection bit, and then you can just add more and more exporters, which just this is like an OpenTelemetry exporter. So there's so much you can do with a map of your network. I think every developer product and every cybersecurity product has a map of your network, so it's the most valuable and flexible kind of data for a development teammate.ADRIANA: Yeah, absolutely. And I'm just spitballing here, but I think a really interesting I wonder if there could be like...a really interesting use case with...I don't know if you're aware of the Open Telemetry demo.ORI: Which one exactly? I guess not.ADRIANA: So there's a repo called Open Telemetry Demo and it's based on the Hipster Shop app, which I believe originated with Google and they've turned it into like a telescope shop now, but it basically showcases instrumentation with OpenTelemetry. So it's this multi-microservices app written in different languages and so it shows different instrumentation in different languages with OpenTelemetry. And it started last year, just before KubeCon, and it's turned into this massively complex, but very cool showcase of OpenTelemetry's capabilities.And it made me think of something where...so recently there's this company called Tracetest. So they're part of this company called Kubeshop. Tracetest is the product and they offer trace-based testing. And so basically the idea is, like, you're already emitting traces. Why don't we take those traces and create test case pieces out of these? And so they recently integrated trace-based testing into the OpenTelemetry Demo to showcase hey, like we can leverage this to basically run automated integration tests.So then I'm thinking, hey, wouldn't it be kind of cool to have Otterize, especially the Network Mapper, integrated into this demo, showcasing, again, OpenTelemetry, the Network Mapper emitting metrics. So anyway...ORI: Yeah, it it a it's a great idea. I wasn't aware of it, but we definitely should do that. And it's funny because most of our tutorials are based on the Google Microservices demo. It's the we still have the stock shop. It's not a telescope shop, but yeah, I guess that app is everywhere. I've seen it on Calico too, and Istio for sure that's a good way to help people discover that the Network Mapper can emit OpenTelemetry metrics. I'm sure it will be useful to a lot of people.ADRIANA: Exactly. And especially it's an open source tool working with another open source tool.ORI: The way God intended.ADRIANA: Yeah, that's exactly. By the way, so we will be releasing this episode during the week of KubeCon. So by the time everyone's listening watching this episode will be out on the Tuesday. Will you be a KubeCon look for the Otterize?ORI: Yes, definitely will be in the main area. Damn. I don't know what our booth number is, but...ADRIANA: Look for the otters.ORI: maybe I can check and you can...yeah...Look for the otters. Maybe I can check and you can edit that in.ADRIANA: Yeah, you know what? I can add it to the show notes once you find the booth name or in booth number, I should say. Do you guys have, like, really cool swag that you're giving out?ORI: You guys, we're going to have the booth other plushies and T shirts, of course. Of course. Oh, my God, the otters are so cute. They have, like, this little key. I wish I had one to show you now.ADRIANA: Damn it. That means you have to go to the booth.ORI: It's like it's been...yeah... You're going to be there too?ADRIANA: Yes. I'm going to be a KubeCon. I'm going to be giving a couple of talks there. There one on platform engineering, actually, and self-service tooling.ORI: Yeah? I'll come see.ADRIANA: So I feel like it's, like, very on topic.ORI: Yeah, I'll be in the crowd. I'll go, "Woo! Yeah!"ADRIANA: I'm going to now make sure that I visit the booth because I definitely want to score one of those otter plushies.ORI: And socks. Will we have socks? Actually don't know. Last time we had socks, they were a huge hit. The first time we had plushies and socks, I was kind of worried, like, would the platform engineers that come to the booth be happy to get those. Maybe it's a bit like, what do grown-ups do with plushies? But then I was like, no, it's actually a hit. And grown-ups have kids too, so not only are they interested in one plushie for themselves, they need also one for every kid so nobody gets there jealous.ADRIANA: Yes, that's right. Because if you bring a plushie home and one of them is not for your kid, you are in trouble. It is the law. I do find the socks are quite popular because I find, like, for me, T-shirts usually are very hit and miss because I'm very petite, and so if it's not very small and fitted, I swim in the T-shirt. So the socks end up working very well because you can't go wrong with socks. They generally fit.ORI: Yeah, and startup socks tend to be like happy socks, very colorful and...ADRIANA: Yes, exactly.ORI: ... maybe you don't want to walk around every day branded, but the socks are like your little secrets.ADRIANA: Yes, exactly. They're perfect. They're perfect. Well, we are just coming up on time, but before we wrap up, are there any parting words of wisdom or hot takes that you would like to share with our audience?ORI: I guess...I'll go for words of wisdom. I'll try, I'm not good on the spot without my...ADRIANA: All right.ORI: ...without my wife, but I guess the thing that I would keep in mind as a developer is to always think about the people that will encounter your code and read your code. A lot of times we think about the design, the performance, does it look good? Does it work well? But I think the most important thing to pay attention to is about the people. And sometimes the people includes you. I'm sure a lot of people watching have come up on a piece of code they wrote and think like, what the hell, what is this? And then find out it's them a couple of years ago.ADRIANA: Oh, my God. It's so true. What the hell was I thinking?ORI: It yeah, and it like, it really goes a long way because, you know, it's the, it's the feeling you get when you use a product and you expect something to be there or you expect to be able to do something, or even the unboxing experience seems like they thought about the person who's opening the product. If you come upon a confusing bit of code that does something nontrivial and you find a comment that says why it's this way and what you should do if you want to change it, like, oh, someone has thought about me.And in some languages, like Go, you can go a bit further and use the compiler to do some of that. I think if you have a function that is doing I/O and it can block, then it's good hygiene to accept a context as a parameter and allow it to be canceled. Because you're not just allowing cancellation, you're communicating, hey, this thing is going to do something that you may want to cancel, it may block. And the cool thing is the compiler then forces the caller to actually make a choice. Like they have to decide which context are they passing in, which is a bit like returning errors. You're saying with a function signature, this thing can fail, which you can't not in every language.You can do that using exceptions, Java, you can say throws and like an endless list of exceptions. But there's nothing fun, more fun than in C getting an unexpected exception.And you're like, what is even going on? How did this get here? And it's like a leaky abstraction. So, yeah, my parting words are think of the people when you write code, not about the machines, because the machines, they're going to be fine. We're going to throw a bit more CPU and memory at it. The people are more important.ADRIANA: I really love that. That those are really great words of wisdom. Well, thank you so much, Ori for geeking out with me today. And y'all do not forget to subscribe. And be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time...ORI: Peace out, and geek out.ADRIANA: Geeking out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the Socials by going to bento.me/geekingout.

About our guest:Tim’s tech career spans over 25 years through various sectors. Tim’s initial journey into tech started in avionics in the US Marine Corps and then into various government contracting roles. After moving to the private sector, Tim worked both in large corporate environments and in small startups, honing his skills in systems administration, automation, architecture, and operations for large cloud-based datastores.Today, Tim leverages his years in operations, DevOps, and Site Reliability Engineering to advise and consult with the open source and cloud computing communities in his current role. Tim is also a competitive Brazilian Jiu-Jitsu practitioner. He is the 2-time American National and is the 5-time Pan American Brazilian Jiu-Jitsu champion in his division.Find our guest on:LinkedInX (Twitter)InstagramFind us on:All of our social channels are on bento.me/geekingoutAdriana’s X (Twitter)Adriana’s MastodonAdriana’s LinkedInAdriana’s InstagramAdriana’s BlueskyShow Links:Transcranial magnetic stimulation (TMS)Generational traumaSelective serotonin reuptake inhibitors (SSRIs)EpilepsyADHDAnxietyDepressionThe Razor’s EdgeNeurodivergentGlimmer vs TriggerBrazilian Jiu Jitsu (BJJ)Psychadelics and mental healthMental Health Resources (Canada)Mental Health Resources (USA)Transcript:ADRIANA: A note to listeners. This week on Geeking Out, we will be talking about mental health issues, including suicide and suicide ideation.Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And geeking out with me today for the second time...I am super happy to welcome back Tim Banks. Welcome back, Tim.TIM: Hey, Adriana. How's it going?ADRIANA: Not too bad. And I'm so excited that you agreed to come back for a second show because of something that you posted online recently that just like I don't know, it kind of got me, like, all verklempt thinking about, like...it was on mental health. And I'll let you open up the conversation.TIM: Sure. And I'm sure there's probably one at the beginning episode, but just the content warning, talk about things like mental health, self harm, suicidal ideation and attempts and stuff like that. So just understand that this is going to be a real and raw. I'd been working on my mental health and been talked about.I've been in therapy and stuff like that, and treatment for depression like medicine and stuff like that. And I've been open about that. But I had a couple of life events happen that kicked off a pretty bad depressive spiral that was already in the middle of a depressive episode that resulted in a suicide attempt at the beginning of July, which was obviously unsuccessful, but only barely.And I don't want to use a wake-up call because it's really more than that. It was like I really have to focus on nothing but my mental health for a while. Like nothing but my mental health. I had to do that. And it was things like having people, my network of friends inside in Tech and outside of Tech, but definitely some folks inside of Tech who, you know...I can't be alone.They were driving like half an hour to come stay at my house for the entire day or two days or three days, right, to I wasn't alone because I couldn't be alone. They were calling, they were sending stuff, they were sending food and things like that. I had my mom come into town and just help me out and just really focusing on nothing but myself, my own mental health. And even that, I mean, that was just the beginning. There was a lot that has to be done.And the reason mostly that that had to be done is because I had kicked that can down the road for so long, right? And there are things I could have been doing, should have been doing, could have been more diligent about or conversations I could have had earlier down the line that would have probably not come to this point. And they say everything happens for a reason, and I'm sure it was, but I would sure hope that I wouldn't have to endure all of this for that reason.For everyone who's listening, I am, night and day dramatically better. So I've been going through treatments called transcranial magnetic stimulation after I finished intensive outpatient DPT therapy, still on my antidepressants and just been really doing a lot of work.And I feel mentally and emotionally better than I have probably any time in my adult life. And I'm almost 50. But the hard part is this was not like a sudden thing.The very last part of it was sudden, right? But the road to get to there was long, right? And as I look back, I realized that there's, like I said, a lot of stuff I should have been doing, should have been focusing on a lot of red flags that I realized now, that I wasn't okay, right?But there's levels of marginalization that sometimes made it difficult to see or recognize or talk about. And it's funny because the one way the patriarchy screws men over specifically, is that we have this weird thing about not talking about our feelings or emotions unless they're anger, but we can't talk about being hurt. We can't talk about being sad or scared or stuff like that. We're socialized to not do that. We're socialized to, quote, unquote, be strong for the family or whatever like that. And that's bullshit.ADRIANA: Yeah, I totally agree.TIM: Can't do that.ADRIANA: Yeah. And I think it's so important for us to have these conversations out in the open because there's so many people suffering in silence on a daily basis. And it's so nice to hear that you had such a wonderful support network when things got really bad for you, that you had people who really wanted to make sure that you were okay and were taken care of, which is so nice. But I think that comes from being open about our mental health issues as well.TIM: Yeah. And I think it was interesting because I was not only open about it to the community, but also to my family, especially my children. Father of five, and I have four under 18 that stay with me from time to time. And this happened while they're with me. I've had conversations about my mental health before, but I tell them I am sick, right?ADRIANA: Yeah.TIM: I'm sick and I can't do the things that I was used to doing or that you would like me to do right now. I can't. And having those conversations when my littler ones are like...they don't really know how to categorize it. They just know that I'm not well. My older ones kind of understood, and I've talked to them more in depth about that kind of things because I want them to understand that they're probably at some point going to go through this.Mental illness can be genetic, or...however passed down it is, it can be passed down. So I don't want them to feel like I don't understand or that I don't want them to ever feel like they can't talk about these things with me, right? But also understand I'm at a very crucially low capacity, critically low capacity.So I'm going to have to ask you to self manage some things or take care of some things or talk to one of my adult friends and your adult friends that we have in your life or other folks like that to help you out. Obviously, as it's gotten better, my capacity has increased and they've noticed. But having that conversation saying, like, I understand you have these expectations, right?This is what I'm capable of delivering right now. And right now capable of delivering is near zero. And being frank about that. And that's the thing when you tell people, it's like, "Hey, I am not okay." I'm not only not okay, but I have been faking and struggling and masking for so long that I have far beyond depleted, right?ADRIANA: Yeah.TIM: And we do that, especially neurodivergent folks. We will a little be the walking wounded. We'll just be held together with bobby pins and bubble gum, just making it through there because we are so used to masking. People who suffer abuse are used to masking, right? You just cover it up and hide it over. And we should probably not do that. But also we have to give people space and safety. These are the buzzwords that actually are important.But people have to have space and safety to feel like they can talk about these things, right? That was really why I was okay talking about at work. I called a couple of friends I had from the hospital...called Kat Cosgrove, who's a good friend of mine, and I was like, "Hey dude, here's the deal..." She was absolutely wonderful.And then I just got a lot of stuff done for...in ways that I was thinking about, "I'm going to lose my job or something's going to happen." And she just mobilized in a way beyond what you would do just for a coworker, but for someone who is a friend. And I cannot be grateful enough to have her and other folks like her in my life.And so I think going out and being that vulnerable and saying to somebody like, "Hey, I really need this help. This is where I'm at and I need this help," and having that person take the ball and like, "I got it."ADRIANA: Yeah.TIM: So building the support networks are important and it's so funny. We talk about in tech, we spend a lot of time and money and have a lot of resources and tooling so we don't have to talk to each other and we should talk to each other more and not about tech. I draw my beat constantly is that tech...the actual tech is the least interesting part about our jobs, right? It is about people. It is about connections, it's about communication.ADRIANA: It is so true. And I think having those lines of communication open with our friends and family is so important. And also, I think back to my parents' generation where talking about mental health issues is so taboo. Like, "Oh, that person's crazy." They're going to be...There's so many negative connotations associated with mental health issues from that generation to the point where then people won't seek the help that they need, and then there's the tendency of passing that mentality down to your kids, so then they sort of see mental health treatment as like, "Oh, no, there's something wrong with me. I'm broken." Okay, so, yeah, you're broken. Fix it.TIM: It it the phrase you're looking for is generational trauma. And that's exactly what it is, especially among marginalized communities or immigrant communities like that. But think about it. Like in our parents' times, they didn't really have mental health care as a thing that you could walk around and have if you were crazy. You went to a place where crazy people went. That's literally how they thought about it, right? And then you were stigmatized forever after that point, right? And so we've come a long way since then as far as the resources and tools that are available to us.But we have come a long way, I mean, objectively in how we talk about it. But also, I don't know if we address it really with the seriousness that it is, because it goes to like, "Hee hee...I'm on antidepressants" as like as a stand up bit, right, to, "Oh, we should watch out for suicide."And there's a long way between those two that we should really be talking about. And so I'd mentioned to you before and mentioned other things that the treatment I underwent, transcranial magnetic stimulation, I only knew about because one of my friends in tech told me about it after.She was like, "Oh yeah, I see what you went through. This really helped me and you should look into it." Right?And that treatment has been life-changing. But I've also talked about meds. Like, I have a conversation about what meds I'm on, and people are like, "Oh, I've done this and look for this," or "This has worked for me." Other folks like, "Oh, you're on this. I should try that."Because we do that for code, right?ADRIANA: Yeah, true. So true.TIM: We do that for literally anything else. Why can't we do that for our own mental health?ADRIANA: Yeah. Absolutely. And it's really just, like, keeping that dialogue open, and I guess...being curious, too, right? Asking the questions, "What are the things that I can do to improve my mental health?" And sometimes that can be like research on your own or asking friends, like posting it out to the community...I think anything like that. To get that information right, we have to arm ourselves with information and try to squash the disinformation, because that's the other thing that will go, right?TIM: It is...it is really bad sometimes. Or the stigma is like, oh, the right wing Twitter, if you're on SSRI, is like, oh, you're on...Like half the country, I feel like is on SSRI sometimes, but it's not really that. It's not that bad. And there are more people that, you know, walking around on mental health medications than you would ever imagine or who are having things to deal with, whether it's epilepsy or whether it's ADHD or whether it's other neurological conditions.That are separate from your kind of traditional mental health things, but still things that they have to deal with that have side effects that have diminishment of their available capacity. And in the end, when we talk about it from a practical standpoint on teams, when my depression and my anxiety are really giving it to me, right, my mental and emotional capacity is exhausted, right? I don't have it. And now that I don't have it, I'm like, "Wow, I've got capacity for days." Right?And so recognizing that as a thing, the practical matters, how can we do as people in business, tech and leadership, you probably want to get a gist of how folks are doing, right?"Hey, how are you doing?""What's your capacity like?"And in a very judgment-free way, it's like, hey, man, look, you don't have to be all in depth of your employees lives and have to request information that they don't necessarily feel comfortable giving you, even if you make it a safe space, right? But you can just say, "What's your capacity?"So that way I know how to task you or give you fewer tasks (more of the point) so that you can focus on stuff. Like, "What's your capacity like?" And if it's low, what can I do to help you?ADRIANA: Yeah, absolutely.TIM: If you just ask those two questions in your 1:1s, right? That's a lot. Like, "What is your capacity?" And if it's low, what can I do to helpADRIANA: And those are definitely conversations that we don't hear enough of. And then you end up in situations where there's a lack of empathy, right, because you don't fully know, well, you know, this person isn't performing up to their full capacity because there's all this other shit going on, right? But you haven't had that conversation. So now it's just like this resentment.TIM: Like, this person should be doing more. They're not doing this or compared to doing whatever, right? And you don't know what they have going on, and they don't necessarily need to tell you, right? Obviously, if someone has a long-term low capacity thing, let's talk about this and see what it is.But there could be...maybe there are a lot of things you can do that involve, hey...that don't involve necessarily, like butting into people's mental health. Burnout is a thing, and it happens, right? And burnout will affect your capacity. So it's like you, as a leader, need to be more mindful of what your team's capacity actually is, right? Instead of what they want it to be, or what it, quote unquote, should be. Of what it actually is, right?ADRIANA: Yeah, absolutely.TIM: And this goes very much into a business and like, leadership kind of aspect, because we run our people at Razor's Edge for, quote, unquote, productivity developers, right? We are pushing for more and more and more and more and more productivity without adding the number of people, mostly, especially when they say, oh, we're in this, quote, unquote, preparing for this economic downturn, which is bullshit, because there isn't one. But but ideally they'll say, "Oh, we need to get this and we need to do this." "We need to meet these numbers," or whatever. But you, as not a manager but as a leader, need to understand what your team's actual capacity is, right? Especially you've been running over or at capacity for an extended period of time. You have to rest, you have to recover, you have to push back on things. Everything can't be the top priority, and that's what leadership is, right?And say, like, "Hey, all right, hey, we really need to ease off the throttle.""We need to come in here, get some maintenance."We need to, like, "Hey, on-calls have been really brutal." We got folks who haven't had vacation in six to eight months, right? We've got folks we need to regroup, we need to let folks rest. We need to put the yoke down for a while, right?Or just eats up...[inaudible]...and go into like, and we have this notion of velocity that's fueled by gamification and metrics that said, "Oh, if we do this fast, this number commits."We have to have more and more and more velocity and more and more and more and more, quote unquote, productivity, right?ADRIANA: And then you eventually hit a wall.TIM: And you lose empathy with it. Yeah. And you can't understand why you keep squeezing this rock and no water comes out.ADRIANA: Yeah, it's so true. And it reminds me of I remember when I was a manager, I was drawing upon...trying to draw upon some of the negative experiences that I experienced, having crappy managers who didn't care about my mental health and vowing that I wasn't going to let that happen to me. Making sure that your direct reports are okay, right? Like asking the question, "Are you okay?"Making sure that if you notice that they haven't taken any vacation for a long time, like, buddy, you need to take some vacation. You absolutely need to. And I think the other one, too, is as a manager, I think it's important to advocate for your direct reports, but also encouraging your direct reports to advocate for themselves. Because sometimes it can be so easy to just ask more and more and more of them and lose sight of the fact that you might be burning them out. And so kind of teaching them, encouraging them to call you out and say, "No, I've had enough. I can't do this anymore."I think it's really important to have those kinds of conversations, and I think it's part of making that safe space with your team as well, so that they do feel comfortable enough so that they can come to you and say, "No, I'm tapped out. I can't do any more."TIM: I think also but it's hard, especially if you're running at diminished capacity. You already have a lot in your plate to summon up, to muster up that gumption, to be like, okay, I need to push back and work. I think we should...early. But when people are in it, I'm all for like, hey, so anybody can advocate for anybody, right?ADRIANA: Yeah. Yeah.TIM: If your work spouse or your best kiki buddy over there at the water cooler or the person you bullshit with around in the group chat, whatever, if they know something's up, you can advocate for that, especially that person has more privilege, has more seniority, has more whatever, more ability to withstand any pushback, right? They'd be like, "Hey, you know what Adriana needs? She's going to need some time. You need to take it easy on her for just a little bit. Like, I'll take some things, assign some out. We can push that. That's not important. That can do it, like, things like that.And then you as a boss have to be like, word, right? And obviously, like I said, if it's for a couple of few weeks, whatever, we can figure out if it's something that's long-term, then there...you know...you can take some time off, take disability, whatever, but like, to understand that that this is not okay, we'll get to it eventually. No, this is acute. This is a need. This has now become a priority, right? When someone is having problems, that is a priority, right? Because no feature you're going to launch is going to be worth more than what could happen to people if we do not give them the time to rest, recover, and focus on their mental health, right?ADRIANA: Yeah.TIM: Somebody's like, oh, yeah, we launched this feature, but three of our developers swallowed bullets. I'm like, okay, wow, I'm so impressed with your velocity. We need to do a better job of putting the people first. And it's not just devs, it's operations folks, it's finance folks. Salespeople, I think, probably are very stress and anxiety-driven, and I think we don't appreciate that enough as sales as developers, because we get salaries, a lot of them get paid on commission. So there's a lot of anxiety around that and there's a lot of pressure on them, right? And a lot of that we can fix. Culturally, fixing the sales culture is probably never going to happen, but we could hope. But that said, you can still, especially as a leader, you can fix that culture in your company, right?You can have realistic stuff, you can have realistic expectations and also leave time for your people to be people, right? And so, I don't know. I am fortunate to be where I was. I was fortunate to be my company at Dell with the people I was working for. But if I was with there are other companies I work for, I know that I would not have had the support that I had here. And it sucks, right? Also, because I know that I've been here, I've been in this almost 50 I've been in the industry now. It's my 27th year. If someone who's junior may not have felt the ability or like, they could say, "Hey, I really need this time release," whatever.So that's why it's important that we who've been there for a while advocate for the other folks that may not be able to advocate for themselves.ADRIANA: Yeah, yeah, I totally agree, because I think back to my very first job out of school, and I worked for a consulting company, and the hours were absolutely brutal, and I hated my life. And I got to the point where I was still living with my parents, and my mom was like, every day, "Are you coming home for dinner?" I'm like, "Mom, I'll tell you when I'm coming home for dinner, because the default answer is no right now, because we're working ridiculous hours."And then I finally hit a point where I'm like, I just cannot take this anymore. And I spoke up for myself. Like, everyone else on the team was dying, but I was just like, I don't care. I'm going to speak up. And I remember getting a bit of a reprieve, but boy, did I feel guilty because I'm, like, the only person complaining about it, which is terrible.TIM: It is and also, there's that thing of, like, leadership by example. Like, if you want to have your people have healthy relationship with work, you should probably have a healthy relationship with work. And I've talked in the past about the idea of divorcing my self worth from my job, which helps a lot, because that way it's like, hey, work is work, and I can like, work, and I love my job, right?But it is not me. It's not who I am. It's just what I do for a living, right? It is not what I would do if I was rich right now, you know what I'm saying? But once you can do that, once you can compartmentalize the things that are they're part of your life, but they're not your life. Once you can compartmentalize those, it gives you clear to focus on that kind of stuff.Like, if you're worried about my career, I was like, I promise you, I don't care so much about my career as I just want to get paid. Right? But I don't have to be a superstar. I don't be a rock star. I just want to make money, right? I make the money. I don't care. You know what I mean?ADRIANA: Yeah, yeah.TIM: And so, that's helped because I know that when I have really been invested at work, like, personally in the politics and all of that is devastating for my mental health.ADRIANA: It's exhausting. Yeah.TIM: Because there's so many things outside of your control. Yeah.ADRIANA: It's so exhausting. I mean, honestly, that's one of the reasons why I went from manager back to IC. I'm like, no thanks. I just want to be off in my little corner doing my thing.TIM: It is great that people can step into those roles, because those are necessary things. But also, you still have to approach them, like I said, with a healthy relationship to work and with your job know?ADRIANA: Yeah, it's so true. It's so true. Yeah. Because you can still get wrapped up in the job regardless of what position you're in. And I can speak for myself. I have been talking about...in my circles, having a healthy having that work-life balance, which is super important to me. But I'm also sometimes the worst at it. I finish work and I can't stop thinking about it.And then something happened to me this summer where my mental health took a toll, where it just got to the point where I was having a physiological reaction to all the things that were accumulating. And then on top of that, I lost my mom last year to cancer, and it's coming up on the one-year anniversary of her death. And so all of these things accumulating, this anniversary coming up, and me just realizing that I've been hustling so hard and haven't had a chance to stop, and constantly thinking about work and body's, like, "Hey, guess what? You're done."I had to take some steps to take care of myself. I promptly found myself a therapist, which I'd been putting off for years because I had a bad relationship with a previous therapist. So that kind of scarred me like, he was an asshole. And so I didn't go to therapy for years, but I'm like, "No, I need a therapist." I need to put processes in place to basically end of the workday. You're done thinking about work.TIM: Close the laptop and done.ADRIANA: Now it's time to think about non worky things. So, yeah, I mean, putting this shit off does not do you any favors whatsoever.TIM: You know what's funny is that my ability to compartmentalize work has been greatly enhanced by working at home than by going into an office. Because when I'm commuting to work, I'm still thinking about work, right? Commuting back from work, I'm still thinking about work. I'm thinking about the whole process of getting ready to go to work. I'm thinking about work, right? But it's like when I walk into my office door here, I'm working. When I close the laptop, I'm not working. And that context switches so fast.ADRIANA: It yeah...it's funny. I don't have a problem with switching off from the working from home. Same thing. Like, just because I'm working from home doesn't mean that...(now I'm not) thinking about the work things.TIM: I look at, like, playing Call of Duty. Like, look, I have an Xbox. Call of Duty is right there. But if I'm not playing Call of Duty, I'm not thinking about Call of Duty, right? You know what I'm saying? I'm not like, OOH, Call of Duty is there to think about. I have a KitchenAid, right? I'm not thinking about baking because my KitchenAid is sitting right there. Right? Okay, cool. I'm not in the kitchen. I don't think about KitchenAid. I'm not even thinking about it. Maybe the context of what am I going to cook tonight? But I'll do that at the time of thinking about it. And so that's why I have this room set apart. Now, obviously, I have the space to have my own office, but you can have your own routine to do that. But I think what's more, the point that I'm trying to make is that one of the things that has helped me out is having these routines, right? Having these boundaries set and then being able to like, hey, this is my boundary. Like, my laptop is closed. I am done. I'm not going to look at this rest of the day.Some people don't have their stuff on their phone. So you'll have two phones, whatever it is. Right. But have boundaries around what you're going to do. And that's not just for work, obviously.That's on other stuff. There's people that like...I am this person, people are going to come to me, and I'm going to solve all the problems and do a lot of emotional labor. And it's difficult for me to say, like, "Hey, I don't have any more"...you know what I mean? I don't have any more capacity left in me. You're going to need to go to somebody else for that.I feel you. That sucks so much. I don't have it right now. I can't pour from an empty cup. And a lot of us do that, especially women with children, right? You all have to do so much. Not that men don't ever, but we all know that women, by and large, have to do a lot of the emotional labor when it comes and physical and manual labor when it comes to the household and taking care of kids and stuff like that. So you're context switching, but you don't ever really get to turn offADRIANA: Yeah, definitely. And I definitely felt that more like when my daughter was really young and I had to have to rush out of the office to pick her up from school, and I'm like, but I haven't finished solving this problem. And then it's like, well, never mind. Time to start job number two, you know? Mom mode enabled.TIM: Yeah, and it's it's harder when they're little. But also, I think one thing that helped me is that, you know, me and my ex, who we co-parent with, we've been very good about raising adults, right? The kids have responsibilities. They have stuff they got to do, like, hey, man, look, you're ten years old. You know how to empty a dishwasher. I'm not doing that.ADRIANA: Yeah, totally.TIM: Right? There you go. Have at it. You want to use the Xbox? Cool. You're going to learn to put the silverware away, and little things like that help. Because in essence, what I'm doing is I need to delegate these things to you because I don't have the capacity to do it all, at least not for a long period of time, right? And a lot of times we feel like we can't do that. Why? Because, oh, you're supposed to be able to do all this. You're supposed to do this, you're supposed to do all that. No, it's supposed to get done. I don't have to do it. Right. There are other people who consume from this who eat at the table. Cool. You can take the plates away. But it's so hard because a lot of us, especially a lot of us who are prone to tech or neurodivergent, like, if I want it done right, I have to do it myself, right? And what we can do is just like, hey, if it's put away, don't stress it, man. If the forks in the spoon thing, it's okay, I promise. If the plates are where the bowls go and the bowls where the plates go, it less than matters. It is so inconsequential, right?And letting those things go, there are some things that matters. Obviously, you don't want to put the knives pointing up. That's dangerous, right? But for other stuff, like, hey, the towels are folded in quarters instead of thirds. Like, oh, okay. Like, fine. You know, like, I don't I'm not going to stress about those. Kinds of things they're put away, right?ADRIANA: Yeah, totally. It's stuff that you don't have to do because someone else did that.TIM: What the real thing that that comes down to is getting perspective and prioritization of, like, concerns, right? Everything can't be important. If everything's a big deal, then you end up doing it all yourself, right?ADRIANA: Yeah, I so agree with you, because I am totally that person who...I will sometimes...like, my husband will refill the dishwasher and I'll rearrange his stuff, and there are times where I'm just like, no, I don't care. That's fine. He's doing it, not me. The whole burden of the world does not have to fall on me. I'm not always successful at it, but I'm trying to train myself because yeah, otherwise you'll just drive yourself mad. You don't have to be perfect at everything, and nothing has to be done your way exactly as you said, it just has to be done.TIM: Like I said, you go and have that conversation, and say, "Look, I'm at capacity, and I can't keep doing this. I'm going to need you to pick up the slack here." I'm all these things I've been doing, I'm going to need you because I know you can, right? Take a little time away from that. Do these things. I need you to pitch in, right? Because they need to get done, right? And then you're asking them to do that. And what you're going to do is not freak the fuck out if something's not exactly how you like it, because it's actually not a big deal.ADRIANA: Yeah, true. And it's about, again, asking for help, because sometimes your other half will be like, "la la la la la"TIM: Oblivious. Yeah.ADRIANA: Yeah, yeah. Right? Whether it's your partner or your kids, sometimes they're just doing their thing. You got to just like, "Can you please do the thing for me? Because I cannot right now."And you know what? They'll say yes, most of the time.TIM: Yeah. And it's like and it's like, you know, kids don't like these chores. I'm like, hey, I need you to take care of this. Or can we or can we get this taken care of? Like, it's going to take a little bit of day. It's really going to help me out because I've got this, this, and this and this. I was like, look, hey, if you want to go here and do this thing, I'll go do this thing, right? And the thing I do is like, hey, if you want to go rewrite this architecture and record this thing, that's fine. I would love for you to do that, right?And I'll take care of the dishes. I feel like there's a skill gap there, and you're fully capable of taking care of these dishes. And it's funny because the way that I reward and incentivize my kids for that and the way I extend them grace when they do things right, that's the hardest thing it is for me to do for myself, right? Extend myself grace because I didn't do this thing exactly right, but it's okay. Or to reward myself in a healthy way for like, hey, man. As a means of recognition for like, yeah, I did this thing, and it was hard, right. A thing that I've been really working on that with my therapist, I really discovered. And when she told me this, I was in tears because it was so impactful.She was like, there's this thing called a glimmer, right? And so I don't know if you've heard of this before, but a glimmer is the opposite of a trigger, right? So where triggers, they trigger like a trauma, like a trauma response or something, like negative, right? A glimmer is like a trigger, but for positive things, for self love or things you love, things that remind you of good stuff, right, or make you feel positive. So I look for like, I have triggers, but I find and make glimmers, right? Sometimes they happen. I'll give you a great example just on the driving home from TMS today, I was, like, singing a song that's a favorite of mine. And one thing TMS has done is made me a better singer and made my penmanship better. And I haven't figured out why, but it does. And so I was like, man, I'm really singing this song. Well, that's really good. I'm a pretty good singer. Little things like that and that just like, yeah, man, I get okay, that's hot. Just little things like that make an impact. So I collect those things and I write them down, right?Keep a little highlight reel or whatever. These are things I've taken the time to recognize in myself, to help myself out, right? Because I feel like we do this really well with other people. Well, to some extent, and not so well with ourselves. We as a culture I think I've said this before, even on here, we as a tech community are awful at recognizing accomplishments.ADRIANA: Oh, my God. Yes.TIM: We're terrible at it, right? Okay. A company might give you a bonus or something like that, but we'll recognize, like, hey, this person is the most impactful, blah, blah, blah, blah. Right? But it's I've been like, people who like, hey, man, they really pulled this thing off, or they were instrumental for this. Like, this thing. We don't recognize the people that chop wood and carry water that well. And we should. And I don't know how we do it, but we should.ADRIANA: Yeah, absolutely. I think we just need to get more in the habit of giving each other kudos.TIM: Yeah, this person's really...Sorry, go ahead.ADRIANA: I was just going to say, I think one of the things that we need to do is just really...in not just celebrating each other, but also ourselves. I know I get into this horrible rut of, like, I accomplished this great thing, right? Like, say I get accepted, my CFP got accepted at Blah Conference. Five minutes of celebration, and then I get depressed right away because I'm like, this is my peak.TIM: Yeah.ADIRANA: I am never going to achieve anything else beyond that. And that's, like, very self-deprecating behavior. So I like your idea of writing down your little pick-me-ups, your little glimmer points, because I think we need to get into these more positive habits. Otherwise, it's so easy to fall into a funk.TIM: It is part of the work I've done, right? And I think it helps is to really recognize, like, oh, man, this is what this means, right? I've watered my plants for, like, a week. I'm like, oh, it's not great little things and recognizing it's like anything else. You know how when we have an outage, we start figuring out how to instrument around that and we have things like, uhoh, this thing is at capacity, this thing is disk full. This thing is page errors. We can do that for ourselves. We can literally have Observability for...ADRIANA: Oh, my God. Yes.TIM: ourselves, and we probably should. Observe the human. And not even just for ourselves have other people, like, hey, my network of people, I have told them, if check in on me and if this is this mention it. Like, bring it up. Like, hey, how are things going? Right? If I haven't gone to Jiu Jitsu in a couple of days, something is wrong.And it's like things like, have, you know, keep the people around you aware of that and keep yourself around that, like logs, journals, whatever. Like, hey, I haven't it's been a few days since I did this. I'd probably need to check in.ADRIANA: Yeah. It's interesting that you mentioned if you haven't gone to Jiu Jitsu for a couple of days, time to check in, because I kind of felt the same way with...rock climbing is my happy place. I love it. It just makes the stress go away. But I had a point a couple of weeks ago, which is when all of the emotions started just crowding in, and I found, like, now, all of a sudden, my safe place, my happy place, was no longer my happy place, and it was a place of stress. And I had to walk away from my happy place for a bit because I was in mental distress. I did not feel like I was in a good place. And when your happy place is threatened like that, I feel like that is like an alarm bell. It's like screaming at you, there is a problem.TIM: Yeah. And it's it's weird because I had that with Jiu Jitsu. Whether it's gym drama or anything else like that, it's like, man, something's going on, because I'm not looking forward to this. And I do. I think that when we talk about going back, that notion of Observability and then touching back about what we talk about what leaders should be doing, there's a level of Observability you have to have into your employees as far as, like, hey, in your 1:1 check-ins, like, "Hey, how are things going? What's your capacity? Like, what can I do to help?" And if you notice that folks are running however you measure task doing or productivity, quote, unquote, or just experience, right? Keep tabs on that.And then be like, hey, I noticed that things aren't going you're kind of a little bit off, and that's fine. It's not a problem yet or anything. I wouldn't say yes, it's not a problem, but I just noticed things are different. I just want to know, see where you are, see what I can do to help, find out what you need, what I can do to help, and giving people that space to do that and also actually doing it, don't just let them say it. You actually got to do it. Helps out a lot. It makes people feel supported.ADRIANA: Yeah, absolutely. And you really nailed it. Like, just giving them space. It's not just like lip service, because I think a lot of companies will pay lip service to mental health issues, but won't actually action anything. And I think that's why it's also so important that you have companies, I think, are embracing more of this idea of taking a mental health day, and that being embraced also through managers who are like, yes, you need to take that mental health day. That is not a problem. You do what you need to do because otherwise you end up with people who are just so burnt out they can't do what they need to do.TIM: Yeah. That has long-term effects in your businesses, whether it's turnover, whether it's not, quote, unquote, productivity or velocity, but how efficient are your practices, how good is the product you're putting out versus how often are you deploying things like that? These things affect your organizational resilience. And we're still fresh off some of the impacts, the worst impacts of the pandemic. And a lot of companies realize they don't have organizational resilience, right?That people are sick, people have problems, and we don't know what to do. We don't have to handle or manage it, especially if we can't observe them from day-to-day at their desks, right? And a lot of companies punted on the difference between management and leadership, and the companies that punted on that are ones that are now we have to return to office. I'm like and if you are a leader or somewhere and you have made the decision that people have to return to office, that is for you, not for your people. Understand that a lot of companies, they work just fine, do amazing work with primarily remote workers, right? Your culture doesn't allow for that. And that's a leadership problem.ADRIANA: Yeah, absolutely. And honestly, it gets me whenever you start hearing these back to the office mandates, because we obviously proved for, what, two, three years that we did a pretty damn good job of working from home. And now it just feels like people are using the excuse of, like, oh, well, you need FaceTime or whatever, just to justify the fact that I spent millions of dollars on a new office and I look like a giant ass.TIM: Sell it.ADRIANA: Right? For real.TIM: Sell the building, rent it out. Yeah, turn it into residential, man. There's a lot of people that need homes right now, you know what I'm saying?ADRIANA: Yeah. So true. Yeah. Even in Toronto, where I'm at...mega, mega housing crisis. And yet you've got all these people, all these organizations with back to the office mandates for these downtown high rises just like, turn them into freaking condo buildings.TIM: You...there's also, like, the other congestion, like, infrastructural problems you create by having. But I think more than anything else, it's also like people who have built an environment at home that really works for them, and then you tell, hey, I need you to come back into this open office. Hard floors, LED lighting, fluorescence, like, no, I'm...not blah desks. I'm not here for that.ADRIANA: Yeah, yeah.TIM: No control over your environment. Yeah, I would rather not. And that will have an impact on people's mental health, like, for folks that are not into talking to everybody. And what I think is so funny is like, well, people got lonely. I'm like, great, those people can come back into the office.ADRIANA: Yep. Yep.TIM: I'm not saying don't have an office, but if you want to go in there, go in there, get around the other people, right? You don't have to force everybody to do it, man. I promise you, right?ADRIANA: Yeah.TIM: If people won't go into the office without you forcing them to come in the office, that should tell you something.ADRIANA: Yep. Oh, yeah. I totally agree. And then I feel bad for the poor folks who are mandated back to the office. Like, oh, you have to go back, like, three days a week, and then they go to the office to go to Zoom meetings. Great. I could have done that at home. Thanks. And now I have to dress fancy to sit at an office for a Zoom meeting.TIM: I think a long story short is that we can go a long way in kind of recognizing where people are with mental health. We go a long way to helping people out with that. But most importantly, the thing we need to do is we need to talk about it and talk about it earnestly and what's in vulnerability and talk about like, this is what works for me, this doesn't work for me, things like that. Maybe I'll have you on for a stream or something like that.One point, because I definitely am going to talk about psychedelics and mental health and how that's helped me. But people have interest in it. But do we talk about it? Because I know when I was coming up it was like, oh, if you do psychedelics, you're going to be some stoned out hippie on the streets. I'm like, I make a lot of money in my job. There are a lot of stigmas we have to break on how people medicate and self-medicate and how they deal with things. And I think that the better environment we set up for folks to have these conversations, the more illuminated we'll all get.ADRIANA: And that's why I'm so very happy that you were able to come on and share your stories and struggles around mental health. Because we need to keep these communications channels open so that more people can feel comfortable about sharing their stories, so we can help each other as a community and destigmatize this whole thing around mental health. Because healthy mind and body equals healthy human.TIM: Yeah. A rising tide lifts all ships. You create a good developer experience by giving them an experience. It has to be a holistic view on developer experience. You can give them all the tools in the world you want, but if they're sad as fuck, right, it doesn't matter.ADRIANA: Yeah. And it's reflected in your organization's outward persona, right? It really is. It's like when you bake angry, it comes out in your baking. Code angry, it comes out in your coding or whatever.TIM: Yeah, I love that because I think about my grandmother when she would argue with my granddad because my grandmother's Mexican when she would argue with my granddad and she would make salsa or chile. We knew as soon as we heard them arguing that we were going to be in for it. At dinner time, we knew we were in for it.ADRIANA: Oh, my God.TIM: Boy, she was cooking mad.ADRIANA: Damn. Well, we are coming up on time, but before we wrap up, do you have any final parting thoughts or advice or anything that you want to share with our audience?TIM: I think, honestly, talk about your mental health struggles as openly as you feel comfortable and if you don't feel comfortable, reach out to some folks who have said that they're open to talking with you. I am always open to it. I may not have the capacity at that time and I'll let you know, but I will be like, hey, yeah, let me talk about I'm happy to hear about it, right? Just talk to somebody. Start there, right. And understand that you are not alone. You don't have to deal with this alone. If your support network is not doing what it needs to do, then I don't want to say get a new support network, but widen your net, right?There are folks out there that will bear the burden with you. For sure.ADRIANA: Yeah, absolutely. Yeah. And that's really important to remember that there are good people out there who are willing to help us out, so making sure that you connect, find those people in your life.TIM: Yeah, absolutely.ADRIANA: Awesome. Well, thank you so much, Tim, for geeking out with me today yet again. Y'all don't forget to subscribe.TIM: Thank you, Adriana. I really appreciate it.ADRIANA: Oh, yeah, no problem. Y'all don't forget to subscribe. And be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time....TIM: Peace out and geek out.ADRIANA: Geeking out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the Socials by going to bento.me/geekingout.If you or someone you know is struggling or in crisis, help is available. In Canada, If you're in immediate danger or need urgent medical support, call 911.If you or someone you know is thinking about suicide, call Talk Suicide Canada at 1-833-456-4566. Support is available 24 hours a day, 7 days a week.For residents of Quebec, call 1-866-277-3553 or visit suicide.ca.In the US, call or text 988 or chat 988lifeline.org . To learn how to get support for mental health, drug, and alcohol issues, visit FindSupport.gov.

About our guest:Tim’s tech career spans over 25 years through various sectors. Tim’s initial journey into tech started in avionics in the US Marine Corps and then into various government contracting roles. After moving to the private sector, Tim worked both in large corporate environments and in small startups, honing his skills in systems administration, automation, architecture, and operations for large cloud-based datastores.Today, Tim leverages his years in operations, DevOps, and Site Reliability Engineering to advise and consult with the open source and cloud computing communities in his current role. Tim is also a competitive Brazilian Jiu-Jitsu practitioner. He is the 2-time American National and is the 5-time Pan American Brazilian Jiu-Jitsu champion in his division.Find our guest on:LinkedInX (Twitter)InstagramFind us on:All of our social channels are on bento.me/geekingoutAdriana’s X (Twitter)Adriana’s MastodonAdriana’s LinkedInAdriana’s InstagramAdriana’s BlueskyShow Links:PerlPythonJohnny-come-latelyPerl for early webcgi-binCustomer Service Manager (CSM)Technical Account Manager (TAM)Business Analyst (BA)Clack fanHashiCorpTerraformMySQLMariaDBPuppetAWS CDKAWS CloudFormationVBasic (Visual Basic)AzureKelsey Hightower on Open Source Pitfalls and ChallengesShay Banon (Founder & CTO of Elastic)MongoDB AtlasWinAmpICQHipChatBEA WebLogicOracleAOLNoOps10x DeveloperLong Snapper (US Football)Quarterback (US Football)Punter (US Football)Transcript:ADRIANA: Hey, y'all! Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada.And geeking out with me today is Tim Banks. Welcome, Tim.TIM: Hey. How's it going, Adriana?ADRIANA: Good. And where are you calling from today, Tim?TIM: I am calling from the balmy summer capital of Austin, Texas, where it's been over 100 degrees for 40 somewhat days straight.ADRIANA: Wow, that is very hot. I am definitely like a tropical gal, but I don't know if I could do 40 degrees over 100 degrees, which I guess is like 40 degrees Celsius for that many days straight. It's a lot. Ouchy. Ouch. Well, stay cool. I hope.TIM: Yeah. Literally, right off camera, I've got my Govee stick fan. Literally, I'm touching it right now, so I have it strategically positioned.ADRIANA: That's awesome. Yeah, nothing like a good fan to help cool off. Yeah, I have one in my office in the summer because it gets like, so hot.TIM: Yeah. I face the window, which is great for the lighting, but also then it gets, like, hot in here, so...ADRIANA: I know, right? Yeah, I have the same problem. All right, well, let's get started with some lightning round questions. All right. I swear they won't hurt. Okay, question number one. Are you a lefty or a righty?TIM: I'm left-handed...ADRIANA: Yay.TIM: but because my dad taught me how to do, like, most sports things, so I throw right handed.ADRIANA: Oh, interesting. That's pretty cool. Yeah, it's funny, there's like, certain things where I think I'm a lefty as well, but I do certain things right-handed. Like, I could not even fathom using a mouse left-handed. That just feels super weird to me.TIM: Yeah, it does.ADRIANA: But I know some left-handed people who are like, yeah, I mouse left-handed.TIM: Yeah. I've switched around. I'm like, that's just weird to me. Right? Same.ADRIANA: Yeah. Also my check marks are right-handed people checkmarks. My mom, who was left-handed, did the left-handed people checkmarks, which were backwards or mirror images?TIM: Yeah, I know. Yeah. It's a pull instead of a push. I still do the push on the check mark, too, because just because.ADRIANA: Yeah, which makes sense because of being left handed, but I think it was like, that conditioning. So anywho yeah, cool. Okay, next question. iPhone or Android?TIM: Oh. iPhone. All day long. All day long. I make good tech money. I don't need a Walmart phone.ADRIANA: Love it. Okay, next one. Mac, Linux or Windows?TIM: Depends on what I'm doing. Windows is never the answer, but I have to use it if I'm gaming. Fortunately, I'm not a PC gamer. I use consoles, so pretty much Mac most of the time. But if I'm doing running a server or something like that, it's going to be Linux.ADRIANA: Fair enough, fair enough. All right, next one. Favorite programming language?TIM: PythonADRIANA: Me too. I love Python. Okay.TIM: It used to be Perl, but then Perl just gets really difficult to read after a while, and I don't have that kind of time anymore.ADRIANA: I've never done Perl, but my job before the current one, they had a lot of Perl and I'm like, damn. I'm not going to lie.TIM: Perl powered Web 1.0. Right? If we're going to be honest, all the cool stuff was done in Perl a little bit. There are some other Johnny-come-lately languages that really got into the later version of the web. But your original cgi-bin stuff, like all the cool stuff yeah. All Perl. That was all Pearl, man. You get it?ADRIANA: I rember that. I never got into Perl, but yeah, that's what I remember it from.TIM: I'm like anyone doing the complex web stuff back in the day was like, it was Perl. Yeah, it's all Pearl.ADRIANA: That's cool. Okay, next question. Dev or Ops?TIM: Oh. Ops. All day long, Ops makes things happen. Look, here's my take on it, and it's going to be spicy, and I don't care, but...ADRIANA: That's awesome.TIM: ...devs are worried about being replaced by AI. Ops is not.ADRIANA: Interesting. I feel like we need to dig into that one a little bit later.TIM: Mm hmm. Oh, we can. We can. I have opinions.ADRIANA: Very intrigued. Next one. JSON or YAML.TIM: JSON.ADRIANA: All right, next one. Do you prefer to consume contents through video or text?TIM: Depends on the content. Right?ADRIANA: Mmmm!TIM: I don't typically like video shorts, so I like little microblogs, but anything of any substance, I like in video because I throw on a video essay and just kind of do whatever. Text is fast. If I want to consume it fast, it's text. If it's going to be anything longer, it's going to be video.ADRIANA: Yeah, that makes sense. And then final question. I just added this one on, which I actually like to ask in interviews. What is your superpower?TIM: Oh, gosh, that's a hard one. So my superpower is understanding people...I think that is a good superpower...kind of digging in and getting behind the eyes a little bit on them and figuring out what's going on. That makes a lot of sense. And I feel like it fits in with the DevRel life because we have as much tech in our work as we do, like, the people side of things. Yeah. Understand. Understanding. Not like I use this, but like, okay, but what are you really trying to do? Right? What is this thing trying to make you help? What is keeping you up at night that this will keep you from keeping keep you from keeping up at night? I guess maybe.ADRIANA: Yeah, that makes a lot of sense. Absolutely. And it's funny because it makes me think of so many times at work where people will come to me and they're like, I need X, and then so I think the gut reaction for a lot of people is like, okay, let's make that happen. But one thing that I've learned over the years is like, but why do you need X? What are you actually trying to do so that you have that extra bit of context? Because maybe you don't need X, maybe you need Y.TIM: So I think the main problem that I've found in software development cycles is that we are so removed from what the customer actually needs and wants.ADRIANA: Right.TIM: Because you think about the customer, how many people that a customer talks to before you actually get to the thing where you're going to develop the thing that you think they're asking for. Right? They're going to talk to a salesperson, they're going to talk to a CSM, they're going to talk to a TAM, they're going to talk to a DevRel. Right? Maybe they talk to company leadership. Right? That gets distilled down and fed to a product manager, who's going to distill something down and feed it to a project manager, who's going to make requirements that you as a developer are going to then try to accomplish. Right. But you don't actually know what the customer is trying to do. And without that context, you're flying in the dark. I've often said that development without context is just masturbation. Right? Because you're doing the thing that you think is right. It makes me feel good, but I don't know if actually no, if it's accomplishing what the other person needs.ADRIANA: Yeah, that's absolutely true. And it's interesting because I think as much as I get annoyed by Agile, I think it tries to bridge that gap a little bit more by getting the right people talking to each other versus the old ways of Waterfall, where I worked at a bank for eleven years and the developers were so far removed from the business people. We had the business people, then we had the business BAs and then we had the technical BAs and then we had the developers and it's like, what the hell?TIM: It...honestly, though, I still feel like Agile and the Sprint system still operates without a lot of context. Right.ADRIANA: Yeah, yeah.TIM: Because we're taking most of our cues from sales, if we're being honest. Right. And that's why I think DevRel is really important because we're the ones that talk to the users and the customers and the community. Right? And so the DevRel should be an engineering role more than anything else because we should be talking directly to product. Product should be talking to us. We should be talking to the engineers. We should be connecting engineers to the people. Like, yes, I love to go up there and give. Well, I don't actually don't love giving. I actually love talking. I love going there, being on panels and stuff like that. But really, who should be at the booths is not DevRel folks. It shouldn't be salespeople. It should be the engineers who are working on the product and the product managers like that. They're the ones that should be talking to the community and to the customers and the users, not salespeople. I mean, yes, your DevRel folks also, but we're going to do that anyways. Right, but if I want to make the most out of an in-person function, I want the people who are building the things that we're selling interacting with the community as much as possible, not the people who are selling. Right? Salespeople at a conference is really not it. It's not it, you know what I'm saying?ADRIANA: Yeah. I know what you mean.TIM: When you walk down to a big city and you go past a place and people are barking at you to buy this thing from the thing that you're really not interested in, that's what that is at a conference. And it's not cool.ADRIANA: Oh my god, that's so true. It's so true. Yeah. And it's interesting because you seldom ever see at conferences...like, the engineers attending. Every so often you will get that, but especially like at a startup where everyone does everything...but at the larger companies you're absolutely right that you see mostly the DevRels or the sales folks at the booth.TIM: And so we need to fix that. Companies like, well, we can only send, like, however many people I'm like, you can send more. You just don't want to, first and foremost. And it's like, you don't need to send salespeople, right? They're going to get the leads from the thing anyways. Right.ADRIANA: Yeah, absolutelyTIM: That's all well and good, the leads that no one's going to respond to. Right. But if you really want to take advantage of being in proximity to the people that use your products or who could potentially use your products, it's the people that build it.ADRIANA: Yeah, it's so true. And it makes a deeper connection because I remember I've worked the booth before doing lhe lead-generation. Like, oh, let me scan your badge. And it's a very impersonal thing. And then when you have your badge scanned, and then you go back to the office after a conference and you're getting all this bullshit spam email, and you're like, oh, my God, just stop. It's irritating. I don't want to talk to you. I'm going to unsubscribe from your list ASAP. Like go away.TIM: Or they give you the drip campaign, which I hey, I know we just talked, like, the four or five replies. I'm like, Bro, I have not replied yet to you, so I'm not sure why you keep emailing me, but it's a sales thing. It's like this thing they teach you... oh, you just keep reaching out. Like, no, man. I don't know why this became a thing. You know who I want to talk to? I don't want to talk to those salespeople, right? If you want to intrigue me on something, let me talk to the person who built it.ADRIANA: Yeah it's so true, it's so true. Talk to the real geeks out there.TIM: I think where we come off the rails and a lot of that development, like I said, is just we operate without context or we think we did something great, this innovative thing, and then the customers are like, yeah, it's not that I want to actually use that thing. It's not that great, or what's going to make the biggest impact. I would have much rather you did this, right? Because we prioritize a feature based on sometimes a specific sale or two or three customers, right? Because it's big numbers right away for these two or three customers. But you could have much bigger numbers if you met the needs of a lot of customers with a feature or with a fix, and then they grow in your product. Right. I don't need to sell a million dollars next quarter. Or what I need to do is I need to sell $20 million over the next three quarters.ADRIANA: I remember I've worked with previous jobs with vendors where they're obviously a small startup and we're obviously their big client. And so they are bending over backwards to get your business and putting certain feature requests on hold for your feature request because you are the big screaming client, which ends up being a detriment to...TIM: Always, always.ADRIANA: the company I think, as a whole.TIM: Because you have that one customer you cannot lose.ADRIANA: Right, yeah, exactly. TIM: And that is a growing pain, because you have now stopped being a company and now you're a contractor.ADRIANA: Yeah.TIM: You're pretty much held hostage by this one client and you have to bend to their will which is a very uncomfortable place to be in. There is this kind of bravery, I would say, almost, that comes...it's actually called leadership...that comes with saying, like, hey, no, this is the vision. We're going to actually develop the product to meet the needs of these customers. If you want it, great. You can buy it. That's awesome. If you don't like it, I want to know why you don't like it, right? And then maybe we can fix it. But also, I'm not going to bend over backwards just for you to the detriment of everything else. Right?ADRIANA: Yeah, absolutely, absolutely.TIM: And so I think the weird part is that people are like, there's a thing that you have to pursue this right now. We need this right now. We read this right now and a lot of people are playing checkers and we need to be playing chess.ADRIANA: Yeah. I love the prop.TIM: I don't know if this is going to be video or audio or not.ADRIANA: It's going to be both, actually So the people that watch this will enjoy the clack fan. I am so happy you brought out the fan actually.TIM: I actually have one that says shade on it, but I left that in the other room. So you just get the rainbow fan. But the point still stands.ADRIANA: Absolutely. But it's interesting on that same point. I think it also brings to mind this whole mentality, especially in corporations. And I think you see this the larger the corporation, where people stop having honest conversations with each other. You're so hell bent on not disrupting the hierarchy and like, oh you went over someone's head. Or people are afraid of owning up to their mistakes because of whatever, makes it very hard to be productive.TIM: There is this...almost politician-like cop-ing out on accountability.ADRIANA: Yes.TIM: No one ever wants to admit a mistake, or no one really wants to admit the true reason behind something, which is just, I want to make money, right?ADRIANA: Exactly.TIM: Let's take HashiCorp, for example, right? They come in here with this, oh, we're protecting open source. And by doing this and people doing this and not contributing...like, no, bro, you had an awful quarter and you're trying to make investors happy. You can just...just say that I would have much more appreciated that than feeding me copious amounts of bullshit about the open source community that you just hung out to dry, right? And in the end, it's going to cost you in the long run. I don't know about a career limiting decision, obviously, but it's certainly going to be a scope-of-influence-limiting decision for HashiCorp over the long run. Right? But they were thinking about this quarter and next quarter, right?ADRIANA: Yeah.TIM: And I'm sure it's going to make some investors are going to be like, okay, great, they're doing something great, but it's not going to be good in the long run. The community is already looking at this and going, all right, well, it's a mistake to rely on a single company for this, so let's either look for an open foundation or do we even need this at all?ADRIANA: Yeah.TIM: Think about it. When Terraform first came out, there wasn't really a good, strong API for interacting with a lot of the ways we consume hardware and infrastructure resources, right? So Terraform is very useful to have something, especially as an alternate to Puppet, right, because it was open source and much more usable and much more elastic, I guess, in those ways, keeping state and being able to compare things like that, committing things to GitHub, etc. etc., right? That's not the case anymore, right? You can have programmatic access to hardware and infrastructure provisioning APIs and health checks, and state have whether you've got CDK and CloudFormation AWS, Google's got whatever it's got. I'm sure Azure's got some VBasic scripts or something like that you can run. I'm kidding. I'm just throwing shade, Azure.But the thing you really have to ask is now, do we need to have another entire product for infrastructure-as-code, which is what it comes down to, or can we now use the native tools and native APIs we have and store state somewhere else? The answer is yes, there are open source alternatives to doing exactly that. And we were never really forced to examine our need for it until Terraform said, ha, we're going to change the license. Right? And then now people can just decide to make you irrelevant, which never would have happened if they had done this. People were just happy on like, okay, cool, we'll keep using Terraform because there's no reason to examine it. Now they have.ADRIANA: Yeah, it's true. And it's interesting because Hashi fans are like, there are hardcore Hashi fans out there, and I do feel like it alienates a lot of them.TIM: It does. And I think the interesting part is that the people who are Hashi fans now, where is your allegiance? Is your allegiance to the company or the product?ADRIANA: Yes, yes...TIM: And those are two different things now, right?ADRIANA: Absolutely.TIM: Or they were two different things and now they're the same.ADRIANA: Yeah, it definitely brings that more to the forefront, as a result.TIM: The...you know, you got a lot of discourse, you know, especially people like Kelsey who are saying, really talking about what does it mean to have open source maintenance, or how do companies, large corporations with capitalistic interests, what influence do they really have on these things? Or how should they be allowed to participate in these open source projects in ways that don't hurt the community, right? And so you really do have to kind of ask it's like, what do we as a community of practitioners and users and influencers and decision makers and things like that, what are we going to do to really change this going forward? Right? And we do have the ability to influence that. We can change that. It's just a matter of are we going to, we're going to dedicate the time and energy to it or are we going to abstract it away like we tend to want to do with everything else?ADRIANA: Yeah, absolutely, or it could be that someone gets pissed off enough, and then... what was the...I want to say there was, like, MySQL that they closed sourced, and then they forked it off to an open source version. Am I getting that right?TIM: Yeah, that was right. Oracle got it and did all that. And then well, I think first it was like when MariaDB first came out, right? Same thing.ADRIANA: YeahTIM: It was just a fork of MySQL, right?ADRIANA: YeahTIM: You look at the same thing with Java.ADRIANA: Yes!TIM: You know, when...whenever a company's been like, okay, cool, we're going to commercialize this thing, and the open source companies be like, I don't think so. I don't think we're going to do that. Before this, it was with Elastic and Amazon with...ADRIANA: Oh yeah, that's right!TIM: ...with OpenSearch, which I really think... now, I'm going to be honest, I know the inside because I used to work for Elastic and I've heard Shay talk at great length sometimes about kind of what his view on that was. And companies owning an open source product is an oxymoron, right? Open source, the community owns it, right.ADRIANA: Yeah, absolutely.TIM: And if you want to dip into that well to make some money, that's fine. Everyone else can too, right? If a company owns it, it's not open source.ADRIANA: That's true.TIM: I mean, they have the ultimate control over it in the end, right?ADRIANA: Yeah.TIM: And we're seeing that over and over again. It's like companies, instead of going through and developing products and service around products and services around this product that make people want to spend money on it, right. They're seeing competition getting upset, and they're trying to stifle competition rather than just getting better at it.ADRIANA: Yeah, yeah, that's true. It's interesting because it makes me think of OpenTelemetry, which is open source, and before, in the Observability world, everyone had their own tracing implementation. Right? And then OpenTelemetry is like, no, let's standardize it. Right?TIM: Yeah.ADRIANA: The cool thing about it is all of the major Observability vendors are fully supporting it, and so you have the community coming together to support this thing, and so what differentiates the vendors is not their proprietary SDKs. It's, how do they render the data? So it changes the conversation, which I think is, like, really cool.TIM: And that's the way it should be. Right? I want to choose a company that offers this service based on the quality of the service, right? Whether it's faster, whether it's more reliable, whether I get better support if I've got these people that do, like, hey, I used to work as a MongoDB, as a service provider, right? And we competed directly with MongoDB Atlas. We competed with AWS to some extent, things like that. I was like, we were just better at it. That was all it was. We were just better at it. Right? And that was fine.But the other thing, too, is..."Better at it." Does that mean that we have enough? Are we essentially like a lifestyle business, or are we trying to make billions of coin, the market, blah, blah, blah? And so I think that's, too, it's like, what is your end goal? Like, there's always this thing. Well, you have to be the biggest and baddest thing. It's like that late stage capitalism thing. No, actually, I'm cool with only owning, like, 20% of the market I'm cool with only owning 10% of the market as long as me and all my employees get paid, right? And customers happy and we do kind of organic growth, that's fine. I don't understand whatever happened to just kind of growing normally and organically? That being okay. Right? We make a profit, people get paid. People can go on vacations. Everyone should be...I don't understand why that was never good enough, right. Or was good enough. And now it's not.But what you end up happening is you have a crush and a grind to get some stuff out, some innovation happens, and as soon as people start trying to return money to investors or go to IPO, now it's the same standard playbook like everyone does. Okay, great. Now we're going to do this, and we're going to do this. We're going to do this. And you see it over and over again. And so all the promise that you see behind a company or a product that gets you excited, you already know that the shoe is going to drop and get short-lived, and now they're going to add like, they're going to ruin it.ADRIANA: Yeah...TIM: Do you remember WinAmp?ADRIANA: Yeah, I do!TIM: Right? WinAmp was great, right? WinAmp, I think it was, like, 2.54. It was like, the thing. It was so amazing.ADRIANA: Yeah.TIM: And do you know what happened to WinAmp?ADRIANA: I don't...no...TIM: They got bought by AOL.ADRIANA: Oh...TIM: and AOL ruined WinAmp.ADRIANA: Oh, jeez.TIM: Yeah, it was awful. I will never forgive them for that. They ruined WinAmp.ADRIANA: Oh, jeez.TIM: Uh, do you remember ICQ?ADRIANA: Yeah, absolutely.TIM: ICQ was great. It is still, to this day, my favorite messaging platform. You know what happened to ICQ?ADRIANA: I don't, no.TIM: They got bought by AOLADRIANA: Get out of town. I did not know that.TIM: Yeah, AOL ruined ICQ. Right?ADRIANA: Oh, man.TIM: Do we see this trend, like, what's happening? A pattern. When we talk about this thing of, we're going to buy this thing and then stops. Do you remember HipChat?ADRIANA: Yeah, I vaguely remember it. I never used it, but I do remember it. Yeah.TIM: And it was kind of cool.ADRIANA: I never use it, but I do remember it. Yeah.TIM: It was kind of fun. And then it got bought. I think it was either Slack bought it from Atlassian and then shut it down.ADRIANA: Interesting. Oh, shoot.TIM: And so when we start seeing this kind of thing where we're like, oh, well, we don't actually want to build a better product, right? We want to eliminate people's ability to choose between products, right? So we don't want to actually have to compete because that's too expensive, right? So it's just cheaper to buy them out. And then not have to make something better, right?ADRIANA: Yeah.TIM: And so that kind of thing is like, I don't like that. Let the community decide if your product is better. Great. People will use it. They'll buy it, right? Or if there's more of a compelling reason for them to use it. If it's less expensive, if it...service is better if it's just more available, whatever it is, right? Let people choose that, right?ADRIANA: Yeah, absolutely.TIM: Don't kill the competition with a pen and lawyers. Kill the competition by building a better product.ADRIANA: Yeah, I totally agree. I totally agree. Yeah. It makes me think of Oracle as well. I spent many years as a Java developer, and so in my Java days, like, the Java app server, my favorite one was, like, BEA WebLogic, and I think Oracle bought them in and was like,TIM: But so many things that we enjoyed have been ruined by acquisition, right?ADRIANA: Yeah, it's super sad.TIM: So when we see this trend continuing through stuff that was used and or beloved by the open source community, it's just more things like, we can't trust corporate interests to have our best interests in mind, because they don't. What's good for the corporation is almost never good for the customer.ADRIANA: Oh, definitely not. Definitely not. Yeah, I feel the same way. I think that's one of the things that frustrated me when I started working was realizing, like, the world is more nefarious than I would like it to be. I'm like, Why can't we all just be friends?TIM: But I think what's worse than that is that there's also the narrative by some straight white dudes on Twitter who are like, oh, well, you're a highly paid capitalist person. You can't be against capitalism. I'm like, just because I detest the system doesn't mean that I'm not good at it, right? Matter of fact, because I'm good at it is why I detest it, right? I'm coming from a place where I know how it works and I know how to make it work for me, and it's terrible.ADRIANA: YeahTIM: I shouldn't have to, right? That's not hypocrisy, that's understanding.ADRIANA: True. Yeah, I totally agree with you.TIM: But I'm not going to mention any names, who it is. They know who they are, and they're probably listening to this.ADRIANA: Now, pivoting...I want to pivot back to a thing that you said earlier when we were talking, when we were doing the rapid fire questions, the Dev versus Ops. You said you had opinions, so I want to hear them.TIM: Yeah. So here's the thing, right? There has been this long, probably a good ten years worth, if not longer... But I know for ten years at least, move to minimize the role of Ops in what we do for software and product delivery, right?ADRIANA: Yeah.TIM: There's DevOps. And they're wrong by thinking this, but a lot of folks think like, oh, it's when dev people do Ops or Ops things gets automated, right? That's not what DevOps is, right? Remember the NoOps push for a while?ADRIANA: Oh, God.TIM: Like, oh, we don't need Ops.ADRIANA: It's like no code. Like, yeah, right.TIM: Yeah. What they're saying is we're abstracting the work of operations away so that devs don't have to be concerned with it, so that they can just do on development. And a lot of developers think that the sun rises and sets by whatever they type out of their fingers and it doesn't, right? The realty is that developers are a skill, don't get me wrong. It's skilled and very difficult. Not very difficult because it's not like any of us are doing brain surgery, right? It is a skilled and it is a highly complex skill set, right? But it is only a piece in the cog, right?And developers are not rock stars, right? They're not. I don't care. The notion of a 10x developer is bullshit, right? You can have all the developers in the world you want to, and you're just going to be developing...you're going to be running stuff on your laptop if it's not for operations, right? Everything happens because of operations, right? And we rely on each other. Like operations need developers to write software for them, they need developers to write drivers, they need developers to write firmware. Like all that kind of has stuff, like APIs, like that also has to work.But when you need stuff to work and you want to make money, it's your operations and support teams that actually do that, right? Developers will get you sales, operations will get you renewals.ADRIANA: Yeah, that's actually a really good point.TIM: And what does any business want more than sales? They want renewals. Let's talk about get renewals from Ops, right? But I think, I think where we come where we come off the rails in a lot of this is like, what are our roles? We're developer relations roles, right? I work for a hardware manufacturer. We don't you know, I should be talking to operations, right? If if you do any kind of API for hardware, stuff like that, you should be talking to operations folks. Unless you're selling specific developer tools, then you should be talking to devs. If you're selling things that people consume in their operations, especially anything around infrastructure or networking, you should probably talk to operations too, right? But we focus on developers, especially when it comes to salaries and stuff like that, because we're thinking, oh, well, operations can be obfuscated away and do this and this and this and this, right?But the operations knowledge gets more and more complex, like how many Ops folks when people talk about using Kubernetes as we abstract away more stuff or like platform engineers like that, they're just operations folks, right?ADRIANA: Yeah, absolutely.TIM: We're providing a platform for developers to use because we do not want them interacting with the hardware, you know?ADRIANA: Yeah, yeah. Sorry. Go ahead.TIM: And so, and as it so there's this...they change the name for operations over and over again, right?ADRIANA: So true.TIM: Because they don't want to just come out and say like, hey, these are Ops folks.ADRIANA: Yeah. I completely agree with you. And it makes me think of something. And to your point, operations folks have been tasked with learning more and more and more stuff, right? Like, picking up managing Kubernetes clusters, and learning these IAC tools and like, you know...TIM: Managing Docker repositories and then CI/CD tools and stuff like that. People who are called DevOps engineers, they're just Ops folks, but they're working on the internal tooling, right? That's all it is, right? But it's funny because really, ideally in the pie in the sky world, the very last concern that a dev is going to have is like, okay, I am now going to package this thing up. That's where it should end. And maybe not even that. I'm just going to commit it to main, right? I'm going to commit it to the production repo. That's where I want you to stop, right? Cool. We've got it the rest of the way, right?Once it gets committed to production, to the production repo, everything else is operations at that point, right? And that's kind of how it should be. I don't want the developers to have to work and not that they can't. I just don't want them to have to I want you to just go back to developing, fixing bugs, developing features, right? Operations has the rest.ADRIANA: Yeah. Now, do you think, though, that it's helpful to have that mutual empathy for what each of these roles entails? Right? Because I do feel like maybe there is this us versus them mentality with Devs versus Ops, and I've been on both sides of it, and I feel like especially when it comes to developers are done developing their feature, throw it over the wall, it's gone into production. That's it. And now there is a bug that arises, like, not my problem.TIM: So I think that's the whole thing that DevOps is supposed to be solving, right?ADRIANA: Yeah, supposed to be...TIM: The throat of the wall, the silent isolation it's supposed to be solving, like they turn into all these other things, and it's not.ADRIANA: Yeah, absolutely.TIM: DevOps is a culture, right? A culture of collaboration and interoperation, right? Where Ops has these levels, concerns and the roles and things they got to do great. It should be informed by what devs do. Devs should be informed by what Ops do. Ops should be able to say like, hey, these are things we found has this problem, this problem, this problem, this problem. So you need to work that into your dev cycle to get these things fixed because these are not operations issues. These are like software bugs or whatever. Right? Cool. That's great. Dev should be able to say like, hey, it's supposed to do this and this and this and this. So when you're designing architecture reviews like that, you have to make sure this and this, it is going to rely heavy on this caching thing. So you have to make sure it's robust. Those are the things that Ops folks should be concerned about with that. Right?I think what ends up happening, and this is just my observation as an Ops person, that Dev has very little knowledge or concern about what happens in Ops. Right. But Ops, by the nature of the role, has to be concerned about the code that's coming out that they have to deploy. Right? And I think to the extent where Devs do not concern themselves with Ops and they think that the whole world revolves around them, that poisons the culture.ADRIANA: Yeah. Absolutely.TIM: And then you end up developing kind of practices around that notion where Dev is the most important thing. Right, but the customers don't interact with the Devs.ADRIANA: No, they don't.TIM: Customers interact with Ops.ADRIANA: Yep, yep.TIM: And that goes back to what I said before about customer-driven development and having context for the things you do.ADRIANA: Yeah, absolutely.TIM: Right, but in the end, I think what's so interesting about that is that we have to have this kind of cultural coming to Jesus moment. But also that's an organizational thing because not all organizations are like that, but a lot of them have come that way and that's where the leadership comes in. Right, but we're not doing leadership, we're doing management. Those aren't the same thing.ADRIANA: And that's where I think having those honest conversations that so many organizations lack becomes a problem as well. Right? You've got people not wanting to admit when there's a problem, people being basically creating their own kingdoms and wanting to protect their own kingdoms and not thinking outside the kingdom. You launch into those sociotechnical issues that plague so many organizations. I definitely see that a lot in big organizations, but it doesn't mean that the small organizations aren't immune to them either. I mean, that can easily happen.TIM: I think that's kind of what you talk about. It's like you talk about how do we do these things? We've created so much tooling and so many products so that we don't actually have to talk one with another, right?ADRIANA: Yes.TIM: So that we try to automate, not even automate empathy. We're just kind of like, oh, we don't need emotions or empathy about this because it's not about the people. So it's like Slack. Why do we have Slack? Because we don't really want to talk to people, we want to be able to chat with them. Right? Why do we have GitHub issues where we interact with people, like over text? But a lot of times too, if you just...not that you can't do this in remote culture, but just like we're doing like, hey man, let's pop into a Zoom meeting for five minutes and we'll talk about this thing. Great. All done. Right.ADRIANA: Yeah.TIM: And yes, I understand the large distributed teams where asynchronous communication is very important, you have to be able to communicate that way, to collaborate. And that's fine. But you understand it's going to be slower, right.ADRIANA: Absolutely.TIM: But if you want to quickly, let's have a conversation. We can understand each other, have some and then I can understand your side better. Or what? Your...not even your side. I can understand what your concerns are better. Right? And have context for why you're saying the things that you're saying or why you have the concerns that you have. And now they can become my concerns, or I can address the concerns that you have a little bit better.ADRIANA: Yeah.TIM: Right, but we have so many tools so that we don't have to do that's.ADRIANA: So true. And I think part of it, too, is people are afraid of confrontation. Like, it's easy to hide behind a chat window, right? But as soon as you're face-to-face with someone, it's it's easier to hate someone or what they're doing behind the chat window. And then the face-to-face is like, there's a person.TIM: Well, I think it's weird because some people rely on other people. Like, that the stereotype of that one asshole engineer that always makes it hard for everybody else. It's a stereotype because it was very often true in a lot of orgs, right? They're relying on getting their way because people don't want to have arguments or confrontation or whatever. Right? And it doesn't have to be about that. It's like, hey, man, you actually don't have that much power. You can argue about all you want to, but once we go for this peer review...in that way, it's good.Right, but at the same time, it's like, if you have to do that just to deal with this one person, how about we just get rid of that one person who doesn't know how to be empathetic and work on a team? Doesn't matter how good he codes, if we have to implement all these things just for this one person.ADRIANA: Absolutely.TIM: You should have organizational fixes, but also you can get rid of toxic people in your org.ADRIANA: Yeah. And that's an interesting point because I think there's this mentality in a lot of teams where, like, oh, but this person is a superstar. We can't possibly get rid of them, but they're fucking with the team's mojo, and that can be worse than having this superstar who knows their shit and fixes things and blah, blah, blah. Because now you're affecting the morale of your other folks, and then that can create tension in-fighting whatever. Right?TIM: Yeah. And we see this like...I'm going to draw a parallel to sports teams. You've seen sports teams that have one superstar player and the rest team is garbage. Right? Or even if they're not garbage, that one superstar player is a problem and it causes discord on the team. Right? They're a problem in the locker room...ADRIANA: AbsolueltyTIM: ...they're a problem with the whole thing. They get so much attention. Negative attention. Right? That the rest of the team falls apart.ADRIANA: Yeah.TIM: And as soon as they get rid of that one person, they're great. Right? But you can have a superstar player that makes other people around them better. Right? You look at like, Michael Jordan, right?ADRIANA: Absolutely.TIM: One of the great examples. Was he a jerk? Yeah. Was he good? Yeah. But did he make other people around him better? Absolutely. Right. And like, that's what you want. You don't want to have somebody who's just, well, this person knows this one technology really great. You know what, I can deal without that. I don't have to have that. If you have to have that, that's a business failureship. That's a leadership failure. Failureship. Failureship.ADRIANA: Yeah.TIM: That's a leadership failure.ADRIANA: I absolutely agree. I think it all boils down to just focusing on the human aspect of...we are humans working with other humans. And we should not forget that. Like, we do not work with robots. Not yet. Maybe someday. Sorry.TIM: So we work on robots, right?ADRIANA: We work on robots. Exactly. Perfect. Yeah.TIM: But technology is inherently still about people. People consume it. People need it. They have needs that they want to address, right? We're basically using these things to solve problems, right? But again, technology for the sake of technology is just masturbation, right? We need to be able to have interoperations of team. We are software engineering teams. We are operations teams. Company is a team, not a family, right? We're a team. And so we have to interoperate.And it's so funny because if you think I'm going to go back to a sports reference for those, you know, it's like everyone on the a lot of people will make fun of the kicker until that kicker needs to save the game, right? And then all of a sudden the kicker is the hero, right?ADRIANA: Yeah.TIM: There are no unimportant parts of a team, right?ADRIANA: Yeah...TIM: People like, "Who's the long snapper?" People know who the star quarterback is, but they don't know who the long snapper is. But the long snapper can lose a game for you. You know what I mean? That's an important thing to remember. The team exists with all the people in it. It's not all about one person. Everybody on that team is necessary and importantADRIANA: Yeah.TIM: whether or not they are rock stars. And then you can extend that beyond just a single team and talk about the interoperability of various teams in an organization like you're. We're about to have some fucking tea right here, my friend. Do you know how sick I am of mostly developers I'm going to go back to that again who are very self-righteous and very degrading to people who are quote unquote, non tech roles like HR, like sales, like customer success.All these people like that. They are why you have a job because of them. I may not agree with having sales pressures, but I will never say we don't need salespeople, right? Because somebody's got to talk to these people because you can't you're not good at that.ADRIANA: That is so true.TIM: That's not your role, right?ADRIANA: So true.TIM: You need to be understand some of these pressures. You need to be able to interface with people of various roles and really go out there and put yourself on the line, right? You have to be able to make sure that we all get paid.You have to make sure that we all have insurance and make sure that we get people in here hired. You have to make sure that your video conferencing stuff works, that you're laptop works.Like all these things are not we don't have anybody that we're not going to say not anybody. For the most part, people are working there because their role is necessary, right?ADRIANA: Yep, absolutely. Things don't happen by magic and fairies.TIM: When people say, oh, the Rockstar, you know what, I've had Rockstar office managers that they paid six figures and absolutely they were worth every penny because they made the stuff happen, right? Yeah, absolutely. I've had Rockstar recruiters everybody at a company. Their role should be important and should be necessary, right? Even if they're the long snapper or the punter.ADRIANA: The yeah, totally. I love that so much. Well, we are coming up on time, but before we finish off and you've given us so many awesome hot takes, but do you have any final hot takes to impart or pieces of advice?TIM: Honestly, the final hot take and piece of advice that I want to impart is to stop having sales-driven development, right? We talked about resume-driven development. We make things overly complex and complicated just because because so and so...and we shouldn't have that but we also shouldn't have development based around a sales quota or based around anything like that. Like, no, man, make the product. Make the product good.ADRIANA: Yeah.TIM: Make the product something that people want to use. Right. And then we'll get it sold. Right.ADRIANA: Totally.TIM: But don't go chasing waterfalls. Just stick to the rivers and the lakes that you're used to.ADRIANA: I love it. I love it. Thank you, Tim, so much for geeking out with me today. And y'all, don't forget to subscribe.TIM: My pleasure.ADRIANA: And be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time...TIM: Peace out and geek out.ADRIANA: Geeking out is hosted and produced by me, Adriana Vilella. I also compose and perform the theme music on my trusty clarinet. Geeking out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the Socials by going to bento.me/geekingout.