AWS Morning Brief

Want to give your ears a break and read this as an article? You’re looking for this link. Never miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of April 19, 2021 with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management: https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/Migrate from Oracle to Amazon Aurora: https://aws.amazon.com/getting-started/hands-on/migrate-oracle-to-amazon-aurora/TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I am Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: We’re coming at you again with some more listener questions from the Unconventional Guide to AWS Cost Management. I’m excited. People are listening to us, Jesse.Jesse: This is fantastic. I’m really excited that we have one fan. I’ve always wanted one fan.Pete: Well, two fans now. Maybe even more because we keep getting questions. And you can also be one of our Friends of the Pod by going to lastweekinaws.com/QA. And you can give us some feedback, you can give us a question and, like, will totally answer it because we like Friends of the Pod.Jesse: We may or may not enter you into a raffle to get a Members Only jacket that’s branded with ‘Friends with the Pod.’Pete: We should get some pins made, maybe.Jesse: Ohh…Pete: I think that's a good idea.Jesse: Yeah.Pete: So, what are we answering today, or attempting to answer for our listener, Jesse?Jesse: So today, we’ve got a really great question from [Godwin 00:01:20]. Thank you, Godwin, Godwin writes, “I truly believe that the system that I support is, like, a data hoarder. We do a lot of data ingestion, we recently did a lift-and-shift of the system to AWS, we use an Oracle database. The question is, how do I segregate the data and start thinking about moving it out of traditional relational databases and into other types of databases? Presently, our method is all types of data goes into a quote-unquote, ‘all-purpose database,’ and the database is growing quite fast. Where should I get started?”Pete: Well, I just want to commend you for a lift-and-shift into Amazon. That’s a Herculean feat, no matter what you’re lifting and shifting over. Hopefully, you have maybe started to decommission those original data centers and you don’t just have more data in twice as many locations.Jesse: [laugh]. But I also want to call out well done for thinking about not just the lift-and-shift, but the next step. I feel like that’s the thing that a lot of people forget about. They think about the lift-and-shift, and then they go, “Awesome. We’re hybrid. We’re in AWS, now. We’re in our data center. We’re good. Case closed.” And they forget that there’s a lot more work to do to modernize all those workloads in AWS, once you’ve lifted and shifted. And this is part of that conversation.Pete: Yeah, that’s a really good point because I know we’ve talked about this in the past, the lift-and-shift shot clock: when you don’t start migrating, start modernizing those applications to take advantage of things that are more cloud-native, the technical debt is really going to start piling up, and the folks that are going to manage that are going to get more burnt out, and it really is going to end poorly. So, the fact you’re starting to think about this now is a great thing. Also, what is available to you now that you’re on AWS is huge compared to a traditional data center.Jesse: Yeah.Pete: And that’s not just talking about the—I don’t even know if I’ve ever counted how many different databases exist on Amazon. I mean, they have a database for, at this point, every type of data. I mean, is there a type of data that they’re going to create, just so that they can create a database to put it into?Jesse: Wouldn’t surprise me at this point.Pete: They’ll find a way [laugh] to come up with that charge on your bill. But when it comes to Oracle, specifically Oracle databases, there’s obviously a big problem in not only the cost of the engine, running the database on a RDS or something to that effect, but you have licensing costs that are added into it as well. Maybe you have a bring-your-own-license or maybe you’re just using the off-the-shelf, but the off-the-shelf, kind of, ‘retail on-demand pricing’ RDS—I’m using air quotes for all these things, but you can’t see that—they will just have the licensing costs baked in as well. So, you’re paying for it—kind of—either way.Jesse: And I think this is something also to think about that we’ll dive into in a minute, but one of the things that a lot of people forget about when they move into AWS says that you’re not just paying for data sitting on a piece of hardware in a data center that’s depreciating, now. You’re paying for storage, you’re paying for I/O costs, you’re paying for data transfer, to Pete’s point, you’re also paying for some of the license as well, potentially. So, there’s lots of different costs associated with keeping an Oracle Database running in AWS. So, that’s actually probably the best place to start thinking about this next step about where to get started. Think about the usage patterns of your data.And this may be something that you need to involve engineering, maybe involve product for if they’re part of these conversations for storage of your product or your feature sets. Think about what are the usage patterns of your data?Pete: Yeah, exactly. Now, you may say to yourself, “Well, we’re on Oracle”—and I’m sure people listening are like, “Well, that’s your problem. You should just move off of Oracle.” And since you can’t go back in time and undo that decision—and the reality is, it probably was a good decision at the time. There’s a lot of businesses, including Amazon, who ran all of their systems on Oracle.And then migrated off of them. Understanding the usage patterns, what type of data is going into Oracle, I think is a big one. Because if you can understand the access patterns of the types of data that are going in, that can help you start peeling off where that data should go. Now, let’s say you’re just pushing all new data created. And we don’t even know what your data is, so we’re going to take some wild assumptions here on what you could possibly do—but more so just giving you homework, really—thinking about the type of data going in, right?

Want to give your ears a break and read this as an article? You’re looking for this link. https://www.lastweekinaws.com/blog/Machine-Learning-is-a-Marvelously-Executed-ScamNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of April 12, 2021 with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I am Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: We’re back again. And we’re here. We made it, Jesse.Jesse: I was worried. This was a journey. Thank you, everybody, for coming on this journey with us.Pete: It was quite an experience going through the Unconventional Guide to AWS Cost Savings. We’ve made it. I just can’t believe we’re here.Jesse: Yeah.Pete: So, what are we talking about today for the culmination of our magnum opus of cost savings optimizations?Jesse: This is a fun one. And I know I keep saying that this is my favorite about everyone, but I have to admit that this one, this topic today probably is my absolute favorite. This one I get really nerdy over. Today, we’re talking about how to predict your future and make your CFO happy. No—spoiler alert—there are not any crystal balls involved in this one. There’s no stock market conversations.This is talking about how you can use all of the different things that we’ve talked about throughout the course of this Unconventional Guide to really bring it all together into a couple ideas that will help you better understand your cloud costs, and really better understand your business, I think.Pete: Yeah. All of the things we talked about really lead up to this one, which is the clients of ours that are the most mature, who are incredibly optimized in their Amazon usage, are the ones who have adopted a majority of these specific items. They all lead to this last one, that ability to predict your future usage based on something that’s happening internally, or if a salesperson comes to you and says, “Hey, we’re about to close this deal, but I need to discount our service.” People are going to start wanting to know well, what is the cheapest that you could sell your service for and still have a positive gross margin?Jesse: Yeah. So, if you’ve done a lot of the things that we’ve talked about in the last couple episodes—I apologize, I know homework’s not the best for a podcast—but if you’ve had the opportunity to work on some of those things, you should have a ton of valuable insights into your spend. We’re talking about tagging, and showback models in particular, maybe even a chargeback model. But you can ultimately use all of this data to better understand what is your forecasted spend is going to look like with a new potential customer coming onto the platform? Or if you get into the topic that we’re going to talk about today, which is mostly unit economics, you can really understand how much can I discount my service and still make a profit, like Pete mentioned?Pete: Yeah, I mean, imagine there’s a global pandemic that happens, and it causes your usage to spike by 500% within the course of a month. How did your spend change? Do you know where it changed? And did it change in ways that you were expecting it to? Like, my databases grew by a lot, and this other thing didn’t grow by very much.Like, that would be expected. But also another thing that—a question that we actually like to ask a lot of our clients, if your sales just doubled overnight, okay would your spend change? Where are the places that are most expensive to operate your service? And again, this is kind of generic. I’ve worked in a lot of SaaS services, so I always think of sales, but just think of whether you’re using the cloud for a SaaS service that you provide and sell, like, B2C, things like that, or B2B, you still have users.They might be internal users. Well, what if your users doubled overnight? What if half the company was using your internal service and now the whole company is? How does that change your usage?Jesse: And it’s also important to think about not just your AWS usage, but all of the other services that you use that support your overall business model: things like monitoring and observability tools, logging vendors, maybe third-party sim tools. All of these are affecting your overall total infrastructure cost and are all part of this conversation. So, it’s really important to start thinking about those architecture diagrams. Remember, when we said, way, way back at the beginning of this conversation, to overlay costs on top of your architecture diagram, understanding that, understanding what parts of your product or what parts of your architecture are the most expensive will really help you understand what’s going to change?Pete: Yeah, let’s say you’ve got a six-figure bill to Datadog or one of the big log management vendors out there, but inside of that bill, is that all just evenly spread across the whole business? What if your log vendor was—the entire spend was all by one service that some developer left the debug logging enabled for? You know, you’d want a way of understanding that maybe that spend was concentrated in maybe a non-production aspect of your account. Because then again, that wouldn’t grow, right? That wouldn’t affect your growth in your sales the same way as if maybe all of your services were equally sending logs of a certain volume over.So, all of those extra services, they all add up, and we see it more and more, as more of our clients start adopting more than just Amazon services: they might be adopting a Snowflake, they might be adopting third-party services running databases running in other services, or EMR type workloads that are not on EMR, and they’re running on Qubole or things like that. There’s just a lot of these services that more and more people are consuming from that fall outside of just the AWS invoice.Jesse: And this also gets back to not just architecture diagrams, but also tagging and showback models, cost visibility, really understanding where your spend is going. And this is fantastic to understand where your spend is going, but finance is probably going to want something a little bit more than this. It’s not just about how much are we spending, or where are we spending it, and maybe it’s not even a finance question. Maybe this is a sales conversation, assuming that you’re a SaaS company. Maybe this is, as Pete mentioned before, “Hey, we want to understand where...

Want to give your ears a break and read this as an article? You’re looking for this link.Never miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of April 5, 2021 with Corey Quinn.

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/Trash Taxi: https://trash.taxiTranscriptCorey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I’m Pete Cheslock.Jesse: I’m Jesse DeRose. [laugh].Pete: Hashtag #FFF. Not my grades in high school; that is Fridays From the Field.Jesse: We will make it a thing. It’s going to happen.Pete: It’s going to happen. We’re going to do our best to use the hashtag triple-F as much as possible. So, if you have any questions for us, just again, reminder, you can go to lastweekinaws.com/QA as we talk more about our Unconventional Guide to AWS Cost Management. Please give us your feedback, ask us some questions, we’ll answer those in a future episode. Today, we’re expanding on tagging. Because it’s so thrilling to talk about tagging some more, Jesse.Jesse: We know that you have struggled to fall asleep at night listening to our podcasts. So, we wanted to do a very special episode just for you, to talk more about tagging. Let’s move into our NPR voices. [silky-smooth voice] Hello, and thank you for listening.Pete: [buttery-smooth voice] Sponsorship of this—no, I’m just kidding. We’re not—we leave that work to, Corey.Jesse: [laugh].Pete: So, today is really about how to win friends and influence DevOps, and it’s all about continual tagging improvement.Jesse: We talked about the importance of tagging, and one of the things that’s really important to tagging is identifying a tagging strategy, and then building and developing that tagging strategy over time. Your tagging strategy is going to change over time; that is the nature of the beast. Your organization is going to change over time, therefore your organization’s needs are going to change over time, and the tagging strategy and the tagging needs are going to change over time, as well.Pete: Exactly. You’re going to build new products; you’re going to grow, hopefully; you’re going to add additional Amazon accounts; you can make acquisitions; you could get sold to another business. There’s just so many things that are going to happen, they’re going to change. It’s just inevitable. So, how do you continue this process of tagging, and this is, I think, a really important discussion because when you start that process, you take that first step and you start investing in tagging, the best way to get those—you know, that compound interest on all of the return value that you’re putting into tagging, is by making it a long term, continual process. And I’m not talking about, like, “Well, you know, we do a little thing every month, and it’ll be good by, I don’t know, maybe a month or two, next quarter. And then we’ll be done.”Jesse: [laugh].Pete: And that doesn’t work. The best companies that we’ve seen that have really knocked this out of the park have turned this into just a multi-year endeavor. It is going to take you a long time to reach just, like, the pinnacle of tagging, having that ability to allocate just down to the penny of your Amazon spend is going to take a long time. So, manage those expectations appropriately that this is not an overnight fix.Jesse: So ultimately, at this point, you’ve tagged all of your resources; you’ve built this policy. The next thing to really think about is, why? Because in a lot of cases, a lot of engineers are going to ask you this very question. Why should we tag this information? Why should we tag these resources?And you’re going to need an answer that’s more than just, “Well, finance wants this information,” or, “Product wants this information,” or, “The engineering leadership team wants this information.” What you’re getting at with tagging is cost attribution. So, at a really high level, for those who aren’t familiar, cost attribution is the process of identifying, aggregating, and assigning your AWS spend to your organization’s teams, your business units, your products, however you want to slice-and-dice that data, whatever different tags you might be leveraging within your tagging policy. So, it’s really about where is your AWS spend going, along these different lines of the different things that finance cares about, that engineering cares about, that product cares about, that IT or security cares about. So, it’s not just about tagging your resources so that everything’s tagged, but it’s about leveraging that information to understand, where are your costs going?Pete: I think that also gives companies a great KPI—Key Performance Indicator for the non-business folks. But it's a good metric. It’s a good way to track your success with tagging is to basically answer this question: what percentage of spend is tagged? Not number of resources because there are some resources that simply don’t have a cost that have the ability to be tagged. So, tracking tagged by a percentage of resources is, for the most, part not useful.Jesse: Yeah.Pete: But tracking what percentage of your spend is tagged—and specifically tags that are enabled as cost allocation tags, which is something that you need to make sure you set up—but by tracking that spend, that KPI, that’s how you can start to understand how good of a job you’re doing at this. Now, again, we’re obviously focused on tags as a cost attribution strategy. But the reality is, is that’s the main use of them on Amazon, specifically. The main use of tags, again, that we see is so people can understand where the money’s going.Jesse: Yeah. AWS even calls them out as user-defined cost allocation tags. For example, if you want to log into Cost Explorer and see where your spend is going among different products, among different teams, among different business units, you need to make sure that those tags that you’re leveraging are enabled as cost allocation tags in Cost Explorer. So, that’s a really important footnote to call out.Pete: Yet to that point, is if you do enable your cost allocation tags, there’s maybe some default ones that Amazon will enable for you, but you’ll have to enable any of your own customs. Those take effect going forward; they’re not retroactive. So, if you want to understand which tag is costing you a certain amount of money, make sure to go and enable that as soon as you possibly can because it’s not going to—you’re goin...

Want to give your ears a break and read this as an article? You’re looking for this link.Never miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill