AWS Morning Brief

AWS Morning Brief for the week of November 2, 2020 with Courtney Wilburn.

LinksLast Week In AWS Twitter: https://twitter.com/lastweekinawshttps://wellarchitectedlabs.com/TranscriptCorey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.Pete: Hello, and welcome to the AWS Morning Brief. I am Pete Cheslock. I'm still here. I'm going to be here for a while I guess, but not alone. I'm here with Jesse. Jesse, thank you again for coming on board and keeping me company.Jesse: Always a pleasure.Pete: It's honestly just nice to talk to someone else that's outside of my little family unit or my pandemic crew.Jesse: I would say it's nice to get paid to just talk about my feelings. But I mean, I'm not technically getting paid for this.Pete: Yeah, I feel like I'm just trying to balance the conversations with coworkers, podcasting this, my kids at this point, have more Zooms than I do.Jesse: [laugh]. I think that probably says something about our social lives and about ourselves. And I feel like I need to go rethink everything.Pete: Well, my son who is six years old, he does a better job of managing his mute button than most full-grown adults I know.Jesse: I feel like that's the fun thing. I really want to see how the next generation is going to grow up with technology, better understanding the mute button, and all of this video content than we do.Pete: It is hilarious to hear my daughter yelling at her friends, “You're on mute.” [laugh]. Oh, well, what is not on mute today is both of us. We are talking about the most loved Amazon service, Amazon QuickSight.Jesse: I think it's technically going to be on blast today rather than on mutes.Pete: Yeah, I think we're going to struggle to keep this one on time. So, if we go long, I apologize in advance. But we're talking about QuickSight, which for those that maybe have never heard of QuickSight before, it's Amazon's business intelligence tool. The question you're probably asking yourself, to be perfectly honest, is why? Why did you even try QuickSight? Like what point, what thing were you solving that made you think of QuickSight? So, we're going to tell that story. But first, let's just pivot into BI tools, business intelligence tools. That's the category that QuickSight is technically in. So, we'll talk a little about that, and also how we actually use BI tools within Duckbill because that'll give you, hopefully, the context into answering that question of, “Why did you even try QuickSight, Pete? Why?”Jesse: I mean, I feel like there's probably still going to be people asking us why after this podcast, and I'm sorry for those listeners. We don't have an answer for you. Maybe we're just masochists. We don't know.Pete: It's just because it's there, I think is what the final answer is. [laugh].Jesse: Absolutely. So, business intelligence tools solve a whole variety of problems and we could probably do an entire episode on them in general. They help you gain insights from your data, which is fantastic. I absolutely love that this is even a category of service out there. But today specifically, to keep it on track, we want to specifically talk about gaining insights from your spend data, your AWS spend data. And to do that, we really need to start by talking about the AWS Cost and Usage Report.Pete: Yeah, the Cost and Usage Report—you might hear it referred to as the CUR. I heard it referred to as the CUR often and it took me quite a while to actually figure out what anyone was talking about. So, if you hear someone say the CUR, they probably mean the Cost and Usage Report. But this is the v2, we'll call it, version of the Amazon billing data. It's incredibly high fidelity, I think is the term. It's very granular; there's a lot of data in there. And it's not enabled by default; you need to actually go turn it on. But what's awesome about this tool is it can provide you some really deep insight into where your money is going, and the only cost for it is the cost to store the data. And the Cost and Usage Report itself, when you turn this report on and have it dumped into your S3 bucket location of choice, you can actually have it store into a couple different file formats. One of them is Excel CSV format. And the other one is a Parquet format, which is a columnar data store and is a lot more efficient for this type of data. And it's the Parquet version of this that we use, we tell our clients—clients of Duckbill—to turn this on and turn it on with Parquet because then you can use tools like Athena to query your data and just leave it in S3 and run those ad hoc queries. So, Athena, though, which we're not talking about Athena, is challenging to use, in some cases—Jesse: Yeah.Pete: —you have to know SQL, which if you don't know SQL you're kind of in a bad spot. So, we use a BI tool, a very popular one called Tableau to query our data on Athena. So, Athena is kind of the engine, you could also obviously put your CUR data into an actual database. But largely, the queries we're doing, these are all human-generated. We're fine if they take seconds; they don't need to happen in milliseconds.Jesse: Yeah, I mean, there's lots of solutions out there. There's third party commercial apps like Tableau and Looker—RIP—there's open-source options like Metabase. But of course then, in true AWS fashion, there's also a hastily integrated acquisition called QuickSight.Pete: So, I have this memory in my head—and hopefully someone will correct me if they're listening to it, and I'm wrong here—but I feel like QuickSight was actually an acquisition. Like Amazon, which really doesn't usually acquire a lot of teams or businesses into Amazon Web Services, with like a couple of pretty rare exceptions, I'm almost positive, that QuickSight was actually some other product that Amazon acquired into it. But the history of QuickSight from at least the Amazon umbrella started around 2015 is when they announced it at re:Invent, and I was there for that announcement. I remember that announcement clearly, and I still actually kind of laugh at it when it came out. Now, first off, that was 2015 is when it was announced, and not for nothing, it does not look like it has gotten much better in the five years that it's been operating since launch. <...

Want to give your ears a break and read this as an article? You’re looking for this link: https://www.lastweekinaws.com/blog/reader-mailbag-savings-plans SponsorsStrongDM: https://strongdm.comLinode: https://www.linode.comNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of October 26, 2020 with Ceora Ford.

LinksLast Week In AWS Twitter: https://twitter.com/lastweekinawsTranscriptCorey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.Pete: Hello, and welcome to AWS Morning Brief. I’m Pete Cheslock. I'm still here; Corey is still not. I'm sorry. But don't worry, I'm here again with Jesse DeRose. Welcome back yet again, Jesse.Jesse: Thank you for having me back. I have to say for all our listeners, I'm sorry I have not watched the entire Step Up trilogy and all the other breakdancing movies we talked about last time. It is still on my todo list. But fear not, it will happen. We will talk about this again.Pete: Well, that actually brings a really good point, which is we need to make a correction from our last podcast. We talked about how Breakin' 2: Electric Boogaloo was the sequel for Breakin’, and I had incorrectly thought that Breakin’—the first one—also had ‘Electric Boogaloo’ in the name. It turns out I lack the ability to read an article on Wikipedia. There was a very carefully placed period in that sentence which, as our listeners probably know, delineates one sentence from another. So, no: Breakin' one, it was just called Breakin’. It was not Breakin’: Electric Boogaloo. I’m—just have no ability to read anything on Wikipedia, apparently.Jesse: I still feel like this is a missed opportunity for the first one in the franchise to be Breakin’: Electric Boogalone.Pete: [laughs]. Almost as bad as Electric Boogalee, but—Jesse: It's up there.Pete: —that's for another podcast. Anyway, we are talking today, not about breakdancing movies from the 1980s, we are actually talking about a little bit of a different change in our normal conversation, not necessarily around Amazon-specific technologies, but around fostering change within an organization, and some of the worst ways that we have seen change kind of implemented into an organization. Fostering change, it's important in any organization in general—and maybe we're a little biased; we spend so much of our time dealing with cost savings and cost optimization, but it really is so much more important when you deal with over-reaching cost optimization and, kind of, management strategy within a company.Jesse: Yeah, I feel like there's this massive disconnect between a lot of companies, where leadership has this really, really heavy incentive—or really, really heavy goal to better understand and manage cloud costs, and the individual contributors or the underlying engineering teams just don't have the same focus. And that's not to say that they don't care about costs, so much as maybe they have other roadmap items that they're working on or other tasks that have been prioritized before cost optimization projects. So, there really seems to be this disconnect to think about cost optimization more thoroughly throughout all levels of an organization. And it ultimately makes us think about how do you go about making that change because it seems like the best way to instill the importance of cloud cost optimization and management across a company is by instilling it in the company's culture. So, today, I really want to focus on what are some of the ways that we can get the entire company to care about cost optimization and management, the same way that leadership might care about cost optimization and management. Or alternatively, if this is an individual contributor that cares, how they can get the rest of the company to care about these things and vice versa.Pete: Yeah, that's a really good point. And we deal with a whole swath of different companies and different people at those companies, where it's kind of amazing to see how some people just inherently really care about what's being spent. And it could be for various reasons. Maybe these are people that may not have any connection to the bill or paying the bill, but more just—they just—I mean, myself, I am this person. I just hate waste. I hate waste in all parts of my life, but I really hate waste in my Amazon bill because finding out that I didn't have to spend $10,000 last month on all of those API list requests on S3 due to that bug, it just—it cuts up my soul.Jesse: And it's really rare to find people in any organization, whether it's a client that we're working with or an organization that you work in, that are super, super invested in that kind of cost optimization work. But when you find them—I was working with one recently at one of our clients who described themselves as a super nerd about cost optimization work. And that's perfect. That's what we want. We want somebody who nerds out over this stuff, and really passionately cares about, what's it going to cost for us to make changes?Pete: Yeah. I mean, we are two people who have focused our careers on caring about how much people spend on their bill. We're cost nerds. It's fine. It's okay to say it.Jesse: I accept this term. I accept.Pete: [laughs]. So, before we get to some of the good ways that we've seen to get people to care about this stuff, we want to talk about some of the worst practices we've seen. And this is broader than just cost management. This really is, what are some of the worst ways that we have been a part of seeing a company just try to affect change, whether you're a startup that's trying to pivot to the next thing, make it to the next funding round; or maybe you're an enterprise and you're just trying to go digitally native, cloud-native, multi-cloud, or something like that. The technology is not your challenge. It's not the technology is the reason why you're not going to accomplish your goal. It's always going to be the people and getting them to care about it. So, what are some ways, Jessie, that you've seen that have been particularly grinding to you?Jesse: Yeah, if we're going to talk about incentivizing practices, I think that the big one that we need to talk about is gamifying the system where the leadership or management sets some kind of goal to say, “We want all of our IT team’s support tickets to be closed within 48 hours.” So, that's a great goal to set; that's a lovely SLA goal to work towards, but if you just set that goal blanketly, for your team, they're going to gamify the system hard. They are going to end up closing tickets as soon as they send a response, rather than waiting for the issue to be resolved or not. I've experienced this multiple times, and it driv...

Want to give your ears a break and read this as an article? You’re looking for this link: https://www.lastweekinaws.com/blog/reader-mailbag-potpourri SponsorsnOps: https://www.nops.io/ Linode: https://www.linode.comNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of October 19, 2020 with guest host Brianna McCullough.

About Corey QuinnOver the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.TranscriptCorey: This episode is sponsored in part by Catchpoint. Look, 80 percent of performance and availability issues don’t occur within your application code in your data center itself. It occurs well outside those boundaries, so it’s difficult to understand what’s actually happening. What Catchpoint does is makes it easier for enterprises to detect, identify, and of course, validate how reachable their application is, and of course, how happy their users are. It helps you get visibility into reachability, availability, performance, reliability, and of course, absorbency, because we’ll throw that one in, too. And it’s used by a bunch of interesting companies you may have heard of, like, you know, Google, Verizon, Oracle—but don’t hold that against them—and many more. To learn more, visit www.catchpoint.com, and tell them Corey sent you; wait for the wince.Pete: Hello, and welcome again to the AWS Morning Brief: Whiteboard Confessional. Corey is still enjoying some wonderful family time with his new addition, so you're still stuck with me, Pete Cheslock. But I am not alone. I have been joined yet again, with my colleague, Jesse DeRose. Welcome back, Jesse.Jesse: Thank you for having me. I will continue to be here until Corey kicks me back off the podcast whenever he returns and figures out that I've locked him out of his office.Pete: We'll just change all the passwords and that'll just solve the problem.Jesse: Perfect.Pete: What we're talking about today is the “AWS Cost Anomaly Detection, Part Two: Electric Boogaloo.”Jesse: Ohh, Electric Boogaloo. I like that. Remind me what that's from. I feel like I've heard that before.Pete: Okay, so I actually went to go look it up because all I remembered was that there was, like, a movie from the past, “Something Two: Electric Boogaloo,” and I dove to the internet—also known as Wikipedia—and I found it it was a movie called Breakin’ 2: Electric Boogaloo], which is a 1984 film. And it says it's a sequel to the 1984 breakdancing film Breakin’: Electric Boogaloo, which I thought was kind of interesting because I always thought of that joke ‘Electric Boogaloo’ was as related to the part two of something, but it turns out it's not. It's actually can be used for both part one and part two.Jesse: I feel like I'm a little disappointed, but now I also have a breakdancing movie from the ’80s to go watch after this podcast.Pete: Absolutely. If this does not get added to your Netflix list, I just—I don't even want to know you anymore.Jesse: [laughs].Pete: What's interesting, though, is that there was a sequel called Rappin’, which says, “Also known as Breakdance 3: Electric Boogalee.”Jesse: Okay, now I just feel like they're grasping at straws.Pete: I wonder if that was also a 1984 film. Like, if all of these came out in the same year. I haven't looked that deep yet.Jesse: I feel like that's a marketing ploy, that somebody literally just sat down and wrote all of these together at once, and then started making the films after the fact.Pete: Exactly. One last point here, because it's too good not to mention, was that it basically says that all these movies, or at least the later one, had an unconnected plot and different lead characters; only Ice-T featured in all three films, which then got me to think a sec—wait a second, Ice-T was in this movie? Why have I not watched this movie?Jesse: Yeah. This sounds like an immediate cult classic. I need to go watch this immediately after this podcast; you need to go watch this.Pete: Exactly. So, anyway, that's the short diversion from our, “AWS Cost Anomaly Detection, Part Two” discussion. So, what did we do last time? Why is this a part two? Hopefully, you have listened to our part one. It was, I thought, quite amazing—but I'm a little bit biased on that one—where we talked about a new service that was very recently announced at Amazon called AWS Cost Anomaly Detection. And this is a free—free service, which is pretty rare in the Amazon ecosystem—that can help you identify anomalies in your spend. So, we got a bit of a preview from some of the Amazon account product owners for this Cost Anomaly Detection, and then we got a chance to just dive into it when it turned on a few weeks ago. And it was pretty basic. It's a basic beta service—they actually list it as beta—and the idea behind this is that it will let you know when you have anomalies in your cost data, primarily increases in your cost data. I remember specifically talking that it was specifically hard to identify decreases in spend as an anomaly. So, right now it only supports increases. So, a few weeks ago, we went into our Duckbill production accounts, turned it on, and we were just waiting for anomalies so that we could do this.Jesse: I also think it's worth noting that I'm actually kind of okay with it being basic for now because if you look at almost any AWS service that exists right now, I would say none of them are basic. So, this is a good place to start and gives AWS opportunities to make it better from here without making it convoluted or difficult to set up in the first place.Pete: A basic Amazon service, much like myself.Jesse: [laughs].Pete: So, guess what? We found anomalies. Well, we didn't find them. The ML backing Cost Anomaly Detection found some anomalies. So, that's what we're here to talk about because now that we actually have some real data, and real things happened, and we actually dove into some of those anomalies, interestingly enough. So, that's what we're here to talk about today.Jesse: It's also probably worth noting that we changed our setup a few times over the course of kicking the tires on this service, and unfortunately, we weren't able to thoroughly test all of the different features that we wanted to test before this recording. So, we do still have some follow up items that we'll talk about at the end of this session. But we did get a chance to look at the majority of options and features of this service, and we'll talk about those today.Pete: So, if you remember—or maybe you don't because you didn't listen to the last episode we did—we configured a monitor, is what it's called, that will analyze your account based on a few different criteria. And the main one is,...

Links MentionedWant to give your ears a break and read this as an article? You’re looking for this link: https://www.lastweekinaws.com/blog/reader-mailbag-accounts/SponsorsStrongDM: https://strongdm.comLinode: https://www.linode.comNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of October 12, 2020 with guest host Veliswa Boya.