AWS Morning Brief

Links:Three vulnerabilities: https://blog.wiz.io/black-hat-2021-aws-cross-account-vulnerabilities-how-isolated-is-your-cloud-environment/Embarrassingly long time: https://Twitter.com/christophetd/status/1486610249045925890“Companies Leave Vast Amounts of Sensitive Data Unprotected”: https://www.propublica.org/article/identity-theft-surged-during-the-pandemic-heres-where-a-lot-of-the-stolen-data-came-from?token=pIt-Qx8lrKMcPei_lM3rFDQpHXkkcxXQGoogle Drive started mistakenly flagging files as infringing copyright: https://www.theregister.com/2022/01/25/google_drive_copyright_infringement/“How to deploy AWS Network Firewall to help protect your network from malware”: https://aws.amazon.com/blogs/security/how-to-deploy-aws-network-firewall-to-help-protect-your-network-from-malware/“How to use tokenization to improve data security and reduce audit scope”: https://aws.amazon.com/blogs/security/how-to-use-tokenization-to-improve-data-security-and-reduce-audit-scope/“Ransomware-resistant backups with S3”: https://www.franzoni.eu/ransomware-resistant-backups/TranscriptCorey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They’ve also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That’s S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.After the content for this episode was effectively laid out, AWS did a late Friday night announcement of a new GuardDuty enhancement that would automatically opt people in to a chargeable service unless they explicitly opted each account out. This obviously doesn’t thrill me or other affected customers. so, as I record this, the situation is still evolving, but rest assured I’m going to have further thoughts on this next week.Now, let’s see what happened last week in AWS security. so, last year, Wiz found three vulnerabilities that allowed attackers to read or write into other customers’ AWS accounts. This flew beneath the radar at the time, but they’re all coming out of the woodwork now, and AWS’s security reputation, more or less, lies in tatters, replaced by a reputation for clamming up and admitting nothing. I’m already wincing at this summer’s re:Inforce keynote. if they try their usual messaging line, it’s not going to end well for them.There was apparently a serious vulnerability within the Linux polkit library. It took Amazon Linux an embarrassingly long time to acknowledge it and put out a release. Now, I’m not a fan of single-vendor Linux installs; any bets on how many non-Amazonians have commit rights to the distribution?Failing to learn from experience is never a great look, but as per ProPublica, “Companies Leave Vast Amounts of Sensitive Data Unprotected” despite decades of breaches. Please, please, please, if you’re listening to this, don’t be one of them. There’s no value in buying the latest whiz-bang vendor software to defend against state-level actors if you’re going to leave the S3 bucket containing the backups open to the world.And an uncomfortable reminder that we might not be the only parties perusing our “private” files stored within various cloud providers, Google Drive started mistakenly flagging files as infringing copyright. Now, amusingly the files in question tended to consist entirely of a single character within the file, but the reminder isn’t usually something that cloud providers want dangled in front of us. Once again we are, in fact, reminded that Google considers privacy to be keeping information between you and Google.Corey: You know the drill: you’re just barely falling asleep and you’re jolted awake by an emergency page. That’s right, it’s your night on call, and this is the bad kind of Call of Duty. The good news is, is that you’ve got New Relic, so you can quickly run down the incident checklist and find the problem. You have an errors inbox that tells you that Lambdas are good, RUM is good, but something’s up in APM. So, you click the error and find the deployment marker where it all began. Dig deeper, there’s another set of errors. What is it? Of course, it’s Kubernetes, starting after an update. You ask that team to roll back and bam, problem solved. That’s the value of combining 16 different monitoring products into a single platform: you can pinpoint issues down to the line of code quickly. That’s why the Dev and Ops teams at DoorDash, GitHub, Epic Games, and more than 14,000 other companies use New Relic. The next late-night call is just waiting to happen, so get New Relic before it starts. And you can get access to the whole New Relic platform at 100 gigabytes of data free, forever, with no credit card. Visit newrelic.com/morningbrief that’s newrelic.com/morningbrief.AWS had a couple interesting blog posts. One of them was “How to deploy AWS Network Firewall to help protect your network from malware”. and I’m torn on this service, to be honest, because On the one hand, it extends the already annoying pricing model of the Managed NAT Gateway, but On the other, it provides a lot more than simple address translation and is cost-competitive with a number of other solutions in this space. I think I’m going to land on, “use it if it makes sense for you, but don’t expect it to be cheap.”And a great blog post from AWS security folks—which is, honestly, something I have said a lot in the past, and I look forward to saying a lot more of in the future—

Want to give your ears a break and read this as an article? You’re looking for this link.https://www.lastweekinaws.com/blog/going-out-to-play-with-the-cdkNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of January 31, 2022 with Corey Quinn.

Links:GitHub organizations: https://alsmola.medium.com/securing-github-organizations-9c33c850638CloudTrail would spew other accounts’ credentials your way: https://onecloudplease.com/blog/security-september-cataclysms-in-the-cloud-formationsSpot on: https://research.nccgroup.com/2022/01/13/10-real-world-stories-of-how-weve-compromised-ci-cd-pipelines/Some excellent points: https://www.darkreading.com/cloud/enterprises-are-sailing-into-a-perfect-storm-of-cloud-risk“Amazon EC2 customers can now use ED25519 keys for authentication with EC2 Instance Connect”: https://aws.amazon.com/about-aws/whats-new/2022/01/ed25519-keys-authentication-ec2-instance-connect/“Integrating AWS Security Hub, IBM Netcool, and ServiceNow, to Secure Large Client Deployments”: https://aws.amazon.com/blogs/apn/integrating-aws-security-hub-ibm-netcool-and-servicenow-to-secure-large-client-deployments/“Best practices for cross-Region aggregation of security findings”: https://aws.amazon.com/blogs/security/best-practices-for-cross-region-aggregation-of-security-findings/Assume AWS IAM Roles using SAML.to in GitHub Actions: https://github.com/saml-to/assume-aws-role-actionTranscriptCorey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They’ve also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That’s S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: So, most interesting this week is probably my request for AWS to support a different breed of SSH key. No, it’s not a joke. Listen on and we’ll get there.So, from the security community last week, everyone talks about how to secure AWS environments. This post takes a different direction and talks about how to secure GitHub organizations, which makes sense if you think about it as an area to focus on. If you compromise an org’s GitHub repositories, it’s basically game over for that company.I also came across this post from 2020, talking about how if asked politely, CloudTrail would spew other accounts’ credentials your way. How many more exploits like this have we seen and just never been told about?NCC Group has some great stories up about compromising CI/CD pipelines, and they are all spot on. Because nobody really thinks about the Jenkins box that has everyone working with it, outsized permissions, and of course, no oversight.Enterprise cloud risk is a very real thing, so a post from Josh Stella, who’s the CEO of Fwage—though he pronounces it as ‘Fugue’—and it makes some excellent points, and also cites me, so of course, I’m going to mention it here. We incentivize the behaviors we want to see more of. There’s a security lesson in there somewhere.Corey: This episode is sponsored in part by our friends atNew Relic. If you’re like most environments, you probably have an incredibly complicated architecture, which means that monitoring it is going to take a dozen different tools. And then we get into the advanced stuff. We all have been there and know that pain, or will learn it shortly, and New Relic wants to change that. They’ve designed everything you need in one platform with pricing that’s simple and straightforward, and that means no more counting hosts. You also can get one user and a hundred gigabytes a month, totally free. To learn more, visitnewrelic.com. Observability made simple.Now, from AWS, what have they said? “Amazon EC2 customers can now use ED25519 keys for authentication with EC2 Instance Connect”. I really wish they’d add support for ECDSA keys as well, and no, this is not me making a joke. Those are the only key types Apple lets you store in the Secure Enclave on Macs that support it, and as a result, you can use that while never exporting the private key. I try very hard to avoid having private key material resident on disk, and that would make it one step easier.“Integrating AWS Security Hub, IBM Netcool, and ServiceNow, to Secure Large Client Deployments”. I keep talking about how if it’s not simple, it’s very hard to secure. AWS, IBM, and ServiceNow, all integrating is about as far from “Simple” as is possible to get.“Best practices for cross-Region aggregation of security findings”. And this was a post that I was about to snark that it should be as simple as “Click the button,” but then I read my post, and to my surprise and yes, delight, it already is. Good work.And in the land of tool, I found a post talking about how to assume AWS IAM Roles using SAML.to in GitHub Actions, and I really wish that that was first-party, but I’ll take what I can get. Because again, I despise the idea of permanent IAM credentials just hanging out in GitHub or on disk or, realistically, anywhere. I like these ephemeral approaches. You can be a lot more dynamic with it and breaching those credentials doesn’t generally result in disaster for everyone. And that’s what happened last week in AWS security.Corey: Thank you for listening to the AWS Morning Brief: Security Edition with the latest in AWS security that actually matters. Please follow AWS Morning Brief on Apple Podcast, Spotify, Overcast—or whereve...

Want to give your ears a break and read this as an article? You’re looking for this link.https://www.lastweekinaws.com/blog/clickopsNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of January 24, 2022 with Corey Quinn.

Links:S3 Bucket Negligence Award: http://saharareporters.com/2022/01/10/exclusive-hacker-breaks-nimc-server-steals-over-three-million-national-identity-numbersAnyone in a VPC, any VPC, anywhere: https://Twitter.com/santosh_ankr/status/1481387630973493251A disgruntled developer corrupts their own NPM libs ‘colors’ and ‘faker’, breaking thousands of apps: https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/“Top ten security best practices for securing backups in AWS”: https://aws.amazon.com/blogs/security/top-10-security-best-practices-for-securing-backups-in-aws/Glue: https://aws.amazon.com/security/security-bulletins/AWS-2022-002/CloudFormation: https://aws.amazon.com/security/security-bulletins/AWS-2022-001/S3-credentials: https://simonwillison.net/2022/Jan/18/weeknotes/TranscriptCorey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.Corey: This episode is sponsored in part by my friends at Thinkst Canary. Most companies find out way too late that they’ve been breached. Thinkst Canary changes this and I love how they do it. Deploy canaries and canary tokens in minutes, and then forget about them. What’s great is then attackers tip their hand by touching them, giving you one alert, when it matters. I use it myself and I only remember this when I get the weekly update with a, “We’re still here, so you’re aware,” from them. It’s glorious. There is zero admin overhead to this, there are effectively no false positives unless I do something foolish. Canaries are deployed and loved on all seven continents. You can check out what people are saying atcanary.love. And, their Kube config canary token is new and completely free as well. You can do an awful lot without paying them a dime, which is one of the things I love about them. It is useful stuff and not a, “Oh, I wish I had money.” It is spectacular. Take a look. That'scanary.love because it’s genuinely rare to find a security product that people talk about in terms of love. It really is a neat thing to see.Canary.love. Thank you to Thinkst Canary for their support of my ridiculous, ridiculous nonsense.Corey: So, yesterday’s episode put the boots to AWS, not so much for the issues that Orca Security uncovered, but rather for its poor communication around the topic. Now that that’s done, let’s look at the more mundane news from last week’s cloud world. Every day is a new page around here, full of opportunity and possibility in equal measure.This week’s S3 Bucket Negligence Award goes to the Nigerian government for exposing millions of their citizens to a third party who most assuredly did not follow coordinated disclosure guidelines. Whoops.There’s an interesting tweet, and exploring it is still unfolding at time of this writing, but it looks that making an API Gateway ‘Private’ doesn’t mean, “To your VPCs,” but rather, “To anyone in a VPC, any VPC, anywhere.” This is evocative of the way that, “Any Authenticated AWS User,” for S3 buckets caused massive permissions issues industry-wide.And a periodic and growing concern is one of software supply chain—which is a fancy way of saying, “We’re all built on giant dependency chains”—what happens when, say, a disgruntled developer corrupts their own NPM libs ‘colors’ and ‘faker’, breaking thousands of apps across the industry, including some of the AWS SDKs? How do we manage that risk? How do we keep developers gruntled?Corey: Are you building cloud applications with a distributed team? Check out Teleport, an open-source identity-aware access proxy for cloud resources. Teleport provides secure access for anything running somewhere behind NAT: SSH servers, Kubernetes clusters, internal web apps, and databases. Teleport gives engineers superpowers.Get access to everything via single sign-on with multi-factor, list and see all of SSH servers, Kubernetes clusters, or databases available to you in one place, and get instant access to them using tools you already have. Teleport ensures best security practices like role-based access, preventing data exfiltration, providing visibility, and ensuring compliance. And best of all, Teleport is open-source and a pleasure to use. Download Teleport at goteleport.com. That’s goteleport.com.AWS had a couple of interesting things. The first is “Top ten security best practices for securing backups in AWS”. People really don’t consider the security implications of their backups anywhere near seriously enough. It’s not ‘live’ but it’s still got—by definition—a full set of your data just waiting to be harvested by nefarious types. Be careful with that.And of course, AWS had two security bulletins, one about its Glue issues, one about its CloudFormation issues. The former allowed cross-account access to other tenants. In theory. In practice, AWS did the responsible thing and kept every access event logged, going back for the full five years of the service’s life. That’s remarkably impressive.And lastly, I found an interesting tool called S3-credentials last week, and what it does is it helps generate tightly-scoped IAM policies that were previously limited to a single S3 bucket, but now are limited to a single prefix within that bucket. You can also make those credential sets incredibly short-lived. More things like this, please. I just tend to over-scope things way too much. And that’s what happened Last Week in AWS: Security. Please feel free to reach out and tell me exactly what my problem is.Corey: Thank you for listening to the AWS Morning Brief: Security Edition with the latest in AWS security that actually matters. Please follow AWS Morning Brief on Apple Podcast, Spotify, Overcast—or wherever the hell it is you find the dulcet tones of my voice—a...

Want to give your ears a break and read this as an article? You’re looking for this link.https://www.lastweekinaws.com/blog/orca-security-aws-and-the-killer-whale-of-a-problemNever miss an episodeJoin the Last Week in AWS newsletterSubscribe wherever you get your podcastsHelp the showLeave a reviewShare your feedbackSubscribe wherever you get your podcastsWhat's Corey up to?Follow Corey on Twitter (@quinnypig)See our recent work at the Duckbill GroupApply to work with Corey and the Duckbill Group to help lower your AWS bill

AWS Morning Brief for the week of January 17, 2021 with Corey Quinn.

Links:Comes with a cryptominer: https://krebsonsecurity.com/2022/01/norton-360-now-comes-with-a-cryptominer/You could be federally charged with wire fraud for paying off a security researcher: https://www.justice.gov/usao-ndca/pr/former-uber-chief-security-officer-face-wire-fraud-charges-0A source code leak of its Azure App Service: https://www.theregister.com/2021/12/24/azure_app_service_not_legit_source_code_leak/“Comprehensive Cyber Security Framework for Primary (Urban) Cooperative Banks (UCBs)”: https://aws.amazon.com/blogs/security/comprehensive-cyber-security-framework-for-primary-urban-cooperative-banks/“Disabling Security Hub controls in a multi account environment”: https://aws.amazon.com/blogs/security/disabling-security-hub-controls-in-a-multi-account-environment/Ipv6-ghost-ship: https://github.com/aidansteele/ipv6-ghost-shipTranscriptCorey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it’s nobody in particular’s job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.This episode is sponsored in part by our friends at Rising Cloud, which I hadn’t heard of before, but they’re doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they’re using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they’re able to wind up taking what you’re running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I’m somewhat skeptical, but their customers seem to really like them, so that’s one of those areas where I really have a hard time being too snarky about it because when you solve a customer’s problem and they get out there in public and say, “We’re solving a problem,” it’s very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it’s worth exploring. So, if you’re looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That’s risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Welcome to Last Week in AWS: Security. Let’s dive in. Norton 360—which sounds like a prelude to an incredibly dorky attempt at the moonwalk—now comes with a cryptominer. You know, the thing that use tools like this to avoid having on your computer? This is apparently to offset how zippy modern computers have gotten, in a direct affront to Norton’s ability to make even maxed-out laptops run like total garbage. Speaking of total garbage, you almost certainly want to use literally any other vendor for this stuff now.“What’s the worst that can happen?” Is sometimes a comforting thought when dealing with professional challenges. If you’re the former Uber CISO, the answer to that question is apparently, “you could be federally charged with wire fraud for paying off a security researcher.”And lastly, Azure continues to have security woes, this time in the form of a source code leak of its Azure App Service. It’s a bad six months and counting to be over in Microsoft-land when it comes to cloud.Let’s take a look what AWS has done. “Comprehensive Cyber Security Framework for Primary (Urban) Cooperative Banks (UCBs)”. This is a perfect case study in what’s wrong with the way we talk about security. First, clicking the link to the report in the blog post threw an error; I had to navigate to the AWS Artifact console and download the PDF manually. Then, the PDF is all of two pages long, as it apparently has an embedded Excel document within it that Preview on my Mac can’t detect. The proper next step is to download Adobe Acrobat for Mac in order to read this, but I’ve given up by this point. This may be the most remarkable case of AWS truly understanding its customer mentality that we’ve seen so far this year.Are you building cloud applications with a distributed team? Check out Teleport, an open-source identity-aware access proxy for cloud resources. Teleport provides secure access for anything running somewhere behind NAT: SSH servers, Kubernetes clusters, internal web apps, and databases. Teleport gives engineers superpowers. Get access to everything via single sign-on with multi-factor, list and see all of SSH servers, Kubernetes clusters, or databases available to you in one place, and get instant access to them using tools you already have. Teleport ensures best security practices like role-based access, preventing data exfiltration, providing visibility, and ensuring compliance. And best of all, Teleport is open-source and a pleasure to use. Download Teleport at goteleport.com. That’s goteleport.com.“Disabling Security Hub controls in a multi account environment”. I hate that this is a solution instead of a native feature, but it’s important. There are some Security Hub controls that are just nonsense. “Oh no, you didn’t encrypt your EBS volumes.” “Oh dear, you haven’t rotated your IAM credentials in 90 days.” “Holy CRAP, the S3 bucket serving static assets to the world is world-readable.” You get the picture.And a tool I found fun, “Port Knocking” is an old security technique in which you attempt to connect to a host on a predetermined sequence of ports. Get it right and you’re now able to connect to the host in question on the port that you want. ipv6-ghost-ship has done something similar yet ever more ridiculous: It takes advantage of the fact that IPv6 means that each EC2 instance gets 281 trillion IP addresses to only accept SSH connections when the last three octets of the IP address on the instance match the time-based authentication code. This is a ridiculous hack, and I love it oh so very much. I’m Chief Cloud Economist at The Duckbill Group, and this has been Last We...