AWS Morning Brief

AWS Morning Brief for the week of March 22, 2021 with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/Pete’s Twitter: https://twitter.com/petecheslockTranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I’m Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: We’re back again. We’re continuing the Unconventional Guide to AWS Cost Savings. What are we talking about this week, Jesse?Jesse: This one’s actually one of my favorite topics. I feel like I say that every episode, but they’re all my favorite topics; just don’t tell any of them that. This week, we are talking about investing in your future. We’re talking about making investments in the AWS platform in terms of reservations.Pete: Awesome, yeah. I mean, there’s usually a return on investment. But investments are a complicated part. I mean, there’s a lot of different ways that Amazon is happy to take your money, right?Jesse: Yeah, absolutely. And I feel like this is one that people are aware of tangentially, but I don’t think a lot of people think about regularly. I really wish more folks would make a habit out of regularly looking at usage and looking at the potential for reservations. Because as you said, Pete, there are amazing opportunities to receive a return on that investment, and I don’t think enough companies are taking advantage of that.Pete: Yeah, there’s a lot of nuances, and we’ll dive into all those things. But before we get started, just want to remind all of our listeners that this Unconventional Guide, you can actually head over to the Duckbill site and go and download this guide, we have it as a handy PDF, for review. Obviously, it’s going to cover some of the future episodes as well. So, you get a little bit of a sneak peek there.Jesse: Spoilers.Pete: But if you do better with a written format, it is available. I would read the link off but it’s comically long and figuring out short URLs, we just haven’t reached that level of technical ability over here. So, we’ll include the link to that PDF in our [show notes 00:02:01], and you can go check it out at duckbillgroup.com. But also to go, too, lastweekinaws.com/QA and ask us questions. Send us your questions, your thoughts, your comments, your feelings. As someone I used to know a long time ago, your bitches, moans, groans, and complaints, just add them all in there. And you can add your name; you don’t have to, you can just send it anonymously. But ask your questions. We’ll be taking some time in future episodes to go into those questions and dive in deeper on some of these particular topics that people might be a little confused by or maybe just want some more insight into.Jesse: Yeah, we’ve gotten some great questions so far that we are planning on future episodes for, and please keep the questions coming. There’s some really, really great questions, really, really great commentary in there. And we absolutely want to make this an engaging conversation. We want this to be a two-way conversation.Pete: Absolutely. So, diving into investments, I’d have to go online and do some research, but I’m pretty sure it was probably the EC2 instance reservations, were the first type of commitment that you can make to Amazon. And again, if I’m wrong, folks out there listening, please go to lastweekinaws.com/QA and let me know of that. Or you could just tweet me as well at @petecheslock. That’s what most people do is, when I’m wrong, it just tweet at me. Right, Jesse?Jesse: Yeah. I mean, well, I have a direct connection to you, but if I didn’t, I’ll just tweet at you.Pete: Yeah, you’ll just tweet at me or Slack DM me or whatever; send me a Zoom message, or maybe hit me up on Chime.Jesse: Oh, god, yes. If somebody is hitting you up on Chime, you know you’re in trouble.Pete: That’s very true. [laugh]. Something has gone wrong if I get a message on Chime. But what’s interesting is that the instance reservations was a way of ensuring capacity, and you could basically commit to running an instance, an availability zone in a certain region, and that instance would be there for you. It was a capacity reservation, which is actually something different now, which we might touch on later, but it wasn’t really like a, “Give me a discount.” That came later. It was an instance reservation: reserve this instance. And this was important because for those folks who have been part of Amazon in the earlier days, there were times that you would ask for a certain instance type in a certain availability zone and Amazon would kindly tell you to go pound sand because they didn’t have one of those for you.Jesse: Yeah, this is something that we’ve seen with a number of clients who are largely multiregional and leveraging basically every instance type you can think of under the sun, and really putting all of these compute resources to their limits. So, getting some kind of confirmation that they would have this capacity available is kind of important.Pete: Exactly. I remember specifically—this was yeah, maybe 2010 timeframe, kind of the heyday, the wild times of Amazon—we had been running—a company of mine had been running a sizable NFS cluster on EC2. “Why would you do that Pete? That’s a terrible idea.” Of course it’s a terrible idea. We didn’t do it by design; we did it because we were a startup, and that was a proof of concept that got out of control, like most technology, right? But when we lost the NFS server itself, we had—I can’t even tell you how many—let’s say 50 EBS volumes that were all striped to this server because that’s a great idea. And we needed another server in that availability zone. We’re not going to snapshot, like, 50 terabytes of EBS. I don’t even know if that capability existed then, to move snapshots across availability zones. So, we needed another instance, and luckily we had a great relationship with our account team—because we were so early—that I do remember, specifically, we got through to the right people. And the line was essentially, “You need to make this API call in the next 15 minutes, or you’re going to lose the instance that we’re basically setting aside f...

Want to give your ears a break and read this as an article? You’re looking for this link.Never miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of March 15, 2021, with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/Building Successful Communities of Practice: https://www.amazon.com/Building-Successful-Communities-Practice-Webber/dp/095749193XTranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: Ever notice how security tends to be one of those things that isn’t particularly welcoming to folks who don’t already have the word ‘security’ somewhere in their job title? Introducing our fix to that, Meanwhile in Security. To sign up for the newsletter or to find the podcast, visit meanwhileinsecurity.com. Coming soon, from The Duckbill Group.Pete: Hello, and welcome to the AWS Morning Brief. This is Fridays From The Field, hashtag-triple-F. I am Pete Cheslock.Jesse: I’m Jesse DeRose, and I have a question: is it hashtag-triple-F, or is it hashtag-F-F-F? Are we spelling out triple F in this hashtag, or is it just literally three Fs?Pete: The three Fs is a little triggering for me for me, with my high school grades, so let’s just stick to—Jesse: [laugh].Pete: —hashtag—Jesse: —triple-F.Pete: Triple-F, I think, just has a better flow to it. But that’s a good—it’s a good point in our continued effort to make triple-F—hashtag-triple-f a thing.Jesse: All of our audience members were really concerned about that one because they’ve been trying to get us trending on Twitter, but they weren’t really sure, was it triple-F. Or was it F-F-F, or was it something in between?Pete: Exactly. It’s just bad. But we’re going to keep trying at it, and we’ll see what happens. Well, anyway, we are back again to continue our Unconventional Guide to Cost Optimization on AWS with another listener question. And unlike the last time we did listener questions, this question actually came in during our Unofficial Guide, which means we actually have one listener this series. Because we can’t count the last one that was from way before. So, to this one listener, thank you, thank you for listening.Jesse: Just that one listener. Just you. Thank you.Pete: Yeah, just you. Everyone else, no, we’re not going to, we’re not going to thank you at all. But if you want to be our second listener, go to lastweekinaws.com/QA and give us a question. What do you want to know more about? What can we dive in a lot deeper on any of these topics we’re talking about? It’s complex stuff, and we’re all learning this, we’re all trying to figure out what works best. And not every company is the same. And that’s what I actually love about this question because this question actually came in from someone who didn’t put their name—but that’s okay—they work in the public sector, which is why they didn’t put their name in there. And they had a pretty interesting question. So, Jesse, maybe you can read this off for us and let us know what we’re going to be answering today.Jesse: Yeah. This question is, “We’re an Azure shop, partly cloud on the way, however, we’re also becoming an Oracle OCI shop”—I’m so sorry—“And an AWS shop, and well, it’s public sector, so one-of-everything cloud provider. How do we convince management that cloud is a different thing than on-prem and needs some kind of cloud team? I dislike the phrase DevOps as a job title, but we need something to change the current model where nearly all of this work is outsourced to a quote-unquote, ‘managed service provider?’” Oof. I have so many feelings.Pete: I would imagine. I mean, I was immediately—I felt called out, you know? Just @ me next time, public sector coward with the DevOps-as-a-job-title phrase.Jesse: Yeah.Pete: They often say that only a DevOps tool, I guess—wait, what’s the term? It’s like, “A DevOps tool would give themselves a DevOps as a job title.” Of course, that’s often said about me because I gave myself a title called ‘DevOps Director’ or ‘Director of DevOps.’ Either way, you phrase it, it’s all pretty bad.Jesse: Yeah. So, there’s a couple of different questions in this, and we’re going to dive into each of them individually. But really, really quick, I want to talk about multi-cloud because that’s kind of the underlying discussion here; something that is not necessarily the focus, but let’s talk about multi-cloud. Why is multi-cloud a thing? Why is it an important thing that you should be thinking about?Pete: Multi-cloud is an interesting topic that could go a lot of different ways. And I call multi-cloud a lot different than hybrid cloud. I think most people are probably doing hybrid cloud, meaning you’ve got some data centers—because it takes you years and years and years to move off of those—and you’ve also got cloud workloads, or maybe you’ve got some data centers and you’re bursting up to cloud workloads; that’s pretty cool, too. I think of multi-cloud as individual applications being deployed to the cloud vendor and cloud provider, based on maybe price or features or things like that. And honestly there, a lot of the cloud providers are getting closer in feature sets. But for example, I might want to use Lambda, but I may not want to suffer high cost of data transfer. So, can I build an application that leverages Lambda, but maybe leverages the extremely low cost of Oracle’s OCI data transfer? That made the news when Zoom signed that big contract with Oracle, it was largely driven by network data transfer. So, there are some reasons why multi-cloud might be a thing.Jesse: And we’ve definitely seen multi-cloud in practice with some of our clients. But I also want to call out the caveat that the clients that were doing this were very mature in their cloud cost practices. So, kudos to those clients because they’re doing amazing, amazing work. But it takes time to really build up a mature, scalable, optimized, multi-cloud strategy.Pete: Yeah, exactly. And I think the biggest challenge is that we see is, on the one hand, if you say to yourself, “I’m going multi-cloud, therefore, I will only consume core primitives like compute, block, store, object store, networking,” even though all the providers will provide you those services, obviously, the APIs to interact with them will be wildly different, but most importantly, the authentication models are going to be wildly different, how you authenticat...

Want to give your ears a break and read this as an article? You’re looking for this link.Join the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of March 8, 2021 with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/AWS Tagging Best Practices Whitepaper: https://docs.aws.amazon.com/whitepapers/latest/tagging-best-practices/welcome.htmlTranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I'm Pete Cheslock.Jesse: I'm Jesse DeRose.Pete: And we're back again, Jesse. We are back. But really have we gone anywhere to begin with?Jesse: We've been making our way slowly but surely through this Unconventional Guide. Lots of really interesting recommendations, lots of really interesting feedback from all of you, which we really, really appreciate. We can't wait to dive into some of those ideas deeper in future episodes.Pete: Yeah. And don't forget, you can give us additional feedback and questions at lastweekinaws.com/QA, feel free to add your name. Or not. Doesn't matter. It can be totally anonymous. That's fine with us. So today, we're talking about a topic that is very near and dear to our hearts. Jesse: Yes.Pete: It is tagging.Jesse: Yes.Pete: Tagging your resources in Amazon, or I mean really any cloud provider; any place you can tag something you probably should. And we're going to talk a little bit about strategies for that, how people use their tags, just all the fun things related to it. Tagging, it's easy to do, right, Jesse? You just tag your resources and all your problems go away.Jesse: Yep. Thanks, everybody, have a good night.Pete: So yeah, if you've enjoyed this podcast, please go to—no, I’m just kidding. Jesse: [laugh].Pete: Tagging is probably the thing that most companies are doing poorly, simply because it's hard, and it's an afterthought, and if you didn't have a really solid forced strategy to ensure tags and force compliance, you're probably not going back to fix it.Jesse: Yeah. It's not thought about as something that's a first-class citizen in the cloud world. When you think about the things that are important to your business model, you might think about getting your application out the door and running, maybe talking about business requirements for availability, failover, data retention, but tagging is nowhere on that list. That's not something that I think any organization thinks about as part of an MVP, let alone future iterations of their products.Pete: Tagging feels much like the same feeling I get when my doctor says that I should eat more veggies. Jesse: Oof.Pete: I know they're good for me; I know we need to do this. They have vitamins, and fiber, and all these wonderful things. But in order to make those veggies something I want to eat, we have to learn to make it more delicious. Personally, I find duck fat works to make them more delicious. I wish we could apply a duck fat strategy to the tagging problem.Jesse: Yeah, it's not an easy problem to solve. Or rather, I should say it is an easy problem to solve, but it's not something that anybody is quickly incentivized to solve. Tagging, just for the sake of tagging, it doesn't work.Pete: Yeah, it's that there really are no incentives for it. No good incentives. It's usually because someone came over to your desk and said, “Hey, what's this charge for? And who's using it? And what's the deal with this?” And you're going into Cost Explorer, and you're like, “Uh, I don't know. It's in this one account.” And that's as far as you can go to figure out who did what and why that thing is the way it is.Jesse: Yeah. There are so many different tagging strategies that we've seen. We've seen some clients talk about tagging as a way to potentially penalize engineers who aren't tagging or who are spending too much money. We've seen organizations who are tagging to reward teams that are tagging all their spend or keeping their spend optimized. Across the board, there are just so many different ways to go about this.Pete: So let's assume you are like most of the companies that we've seen. Definitely not all: there are some rare gems out there that are making tagging a long term and continual process, which we're actually going to talk about in a future episode, how to do that. But let's say you're just looking at your bill, you're looking at your usage, and you're saying to yourself, “Okay. I need to be better at this.” What do they say, “The journey of a thousand miles starts with a single step?” What is that first step?Jesse: Yeah there's a lot of different ways to go about this. I think there's a couple great places to start. Now, I will say AWS has a thrilling 24-page best practices white paper that we’ll throw a link in the [show notes 00:05:18]. Pete: Have you read that, Jesse?Jesse: I will say that I have read parts of it. I have not read all of it, and so I want to make it very, very clear to all of our listeners, this is not a document that needs to become the holy grail for your organization. I think in the same way that you could read the SRE book from Google and have some good takeaways, you can skim through this white paper, maybe read through a couple of the sections that seem most applicable to your organization, and then start with those ideas, start with those best practices, and then build them over time organically; develop them over time organically.Pete: I like to read it some nights when I'm just having trouble sleeping, and maybe by page two or three I’m just out.Jesse: Yeah. There's a lot of content in there talking about what to tag, why to tag. I think the best place for any organization to start is to think about what are the important things that we need to tag. And that's a conversation that's going to involve not just engineers, but also finance, potentially IT, maybe also security teams, depending on how your organization is built. Because ultimately, what you want to do is understand what are the things that my organization cares about when it comes to our cloud usage?&n...

Want to give your ears a break and read this as an article? You’re looking for this link.Never miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of March 1, 2021 with Corey Quinn.