AWS Bites

Should I use CloudFormation or should I use Terraform instead? If you are just starting to do Infrastructure as Code (IaaC) you probably have this question. In this episode we will discuss in detail how these two amazing pieces of technology compare against each other and what their features, weaknesses and strengths are. We will share our opinions based on our experience with these 2 technologies and guess what, for once we have a bit of clash of opinions! Can you guess who is in the Terraform camp and who is in the CloudFormation camp instead? In this episode, we mentioned the following resources: - A tutorial on how to create resources conditionally with CDK (and CloudFormation): https://loige.co/create-resources-conditionally-with-cdk - An article to understand in depth how to use secrets management with SSM and SecretsManager together with CloudFormation: https://dev.to/eoinsha/3-ways-to-read-ssm-parameters-4555 - Ben Kehoe’s tweet about switching from CloudFormation to Terraform: https://twitter.com/ben11kehoe/status/1158758917515763712 - Terraform null resources: https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource - CloudFormation Macros: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/macros-example.html - How to workaround missing CloudFormation features (by Cloudonaut): https://cloudonaut.io/three-and-a-half-ways-to-workaround-missing-cloudformation-support/ - Org-formation: https://github.com/org-formation/org-formation-cli - How to create accounts in an org with Terraform: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_account - Control Tower Account Factory for Terraform: https://learn.hashicorp.com/tutorials/terraform/aws-control-tower-aft - Pulumi: https://www.pulumi.com/ - Cloudonaut’s comparison of CloudFormation with Terraform: https://cloudonaut.io/cloudformation-vs-terraform/ - Cloudonaut’s free CloudFormation templates: https://templates.cloudonaut.io/en/stable/ Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

AWS Lambda just got a big upgrade in ephemeral storage: you can now have up to 10 GB of storage for your /tmp folder! Before this was limited to “only” 512 Mb… But is this really useful? What can we do now that we couldn’t do before? Also, is this going to have an impact on price? And how does it compare with other storage capabilities that are available in Lambda? Eoin and Luciano are on the case to try to find some answers to these compelling questions, for the greater serverless good! In this episode, we mentioned the following resources: - Official AWS announcement blog post: https://aws.amazon.com/about-aws/whats-new/2022/03/aws-lambda-configure-ephemeral-storage/ - Will Dady on Twitter about 10GB of ephemeral storage now enabling interesting CI/CD use cases: https://twitter.com/WillDady/status/1507110176209322018 - Yan Cui’s post on Lumigo’s blog “Welcome to 10GB of tmp storage with Lambda”: https://lumigo.io/blog/welcome-to-10gb-of-tmp-storage-with-lambda/ - Lambda cost comparison with ephemeral storage spreadsheet: https://docs.google.com/spreadsheets/d/1_oGjLN0BeRR8CWfgdjeYiIknRTugdmJOhGkAjLCTStw/edit?usp=sharing This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Eoin and Luciano take you through the ways serverless can give you more security out of the box. We cover the tradeoffs between having more security control and the responsibility that comes with this power. There are always new security challenges so we cover some of the common pitfalls with serverless and AWS security in general. Finally, we share some tips to make your serverless deployments more secure. In this episode, we mentioned the following resources: Architecting Secure Serverless Applications on the AWS Architecture Blog: https://aws.amazon.com/blogs/architecture/architecting-secure-serverless-applications/ AWS IAM Access Analyzer: https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.htm The AWS response to the Log4J2 vulnerability: https://aws.amazon.com/security/security-bulletins/AWS-2021-006/ FunctionShield: https://github.com/puresec/FunctionShield This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Luciano and Eoin discuss their strategies and ideas to help new team members to start embracing cloud computing and get productive with AWS. What are the main concepts to focus on when bootstrapping this journey, how to make a plan and make sure it’s bespoke to the expectation of the new employee. How to do pairing sessions and make sure we can build hands-on experience. Finally we discuss building troubleshooting skills and make sure we put in place a virtuous cycle that can foster continuous learning. In this episode, we mentioned the following resources: - Our previous episode about AWS certifications and learning material: https://www.youtube.com/watch?v=qf0CuUOtPEI This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Eoin and Luciano try to answer the question of what AWS accounts do you need for your team and how to organize them. In this episode of AWS bites we discuss common ways to organize AWS accounts and environments from the perspective of a company running production workloads. We try to answer questions like “how many accounts and how many environments?”. We also discuss how you and your team can be more productive by effectively managing AWS accounts and environments. Finally we explore some common security and cost-related tradeoffs that are common when it comes to organizing AWS accounts. Thanks to David Lynam for suggesting this awesome topic! In this episode, we mentioned the following resources: - AWS Account vending machines: https://aws.amazon.com/solutions/implementations/aws-landing-zone/ - Org Formation: https://github.com/org-formation/org-formation-cli - Terraform for accounts and organizations: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_account This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Luciano and Eoin explore the wonderful world of data streaming using Kafka on AWS. In this episode we focus mainly on Managed Streaming for Kafka (or MSK) and discuss what are the main differences between MSK and Kinesis. We also explore the main features that MSK provides, its scaling characteristics, pricing and, finally, how MSK works in conjunction with other AWS services. We conclude the episode by providing a decision tree that should help you to decide whether you should use Kinesis or MSK or avoid streaming services entirely in favor of something like SNS or SQS. In this episode, we mentioned the following resources: - Our previous episode on Kinesis data streams: https://www.youtube.com/watch?v=u_nR6up4Kvs - Our series of Event services: https://www.youtube.com/watch?v=CG7uhkKftoY&list=PLAWXFhe0N1vLHkGO1ZIWW_SZpturHBiE_ - AWS MSK sizing spreadsheet: https://dy7oqpxkwhskb.cloudfront.net/MSK_Sizing_Pricing.xlsx - Should My Startup use Kinesis or MSK? - https://www.youtube.com/watch?v=TJS19EuzH2k - Intro to MSK (reinvent talk from 2018) - https://www.youtube.com/watch?v=9nKswHsLseY - Running Apache Kafka on AWS (by Frank Munz) - https://www.youtube.com/watch?v=HtU9pb18g5Q - Cloudonaut - Kinesis versus MSK - https://www.youtube.com/watch?v=kcBAKz0MPf8 This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Eoin and Luciano are back with some more AWS messaging services material! This time we talk about Kinesis Data Streams and everything there’s to know about them! In this episode of AWS Bites, we cover the difference between data streaming and queue or message buses. How Kinesis can let you process large batches of messages in near real time. Why you might use Kinesis as a queue or pub/sub bus in some cases. A few different ways to use Kinesis with the focus on avoiding too much complexity. We also provide some insights based on our experience with Kinesis Data Streams including real use cases, monitoring tips and some gotchas to be aware of. In this episode, we mentioned the following resources: - Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY - Amazon Kinesis Producer Library (KPL): https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html - Amazon Kinesis Consumer Library (KCL): https://docs.aws.amazon.com/streams/latest/dev/shared-throughput-kcl-consumers.html - Kinesis Library wrapper for Node.js: https://github.com/awslabs/amazon-kinesis-client-nodejs - Kinesis Library wrapper for Python: https://github.com/awslabs/amazon-kinesis-client-python - Kinesis Data Stream with captures from DynamoDB (for CDC): https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html - Kinesis Data Stream with captures from Aurora: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Overview.html - Kinesis Data Analytics: https://aws.amazon.com/kinesis/data-analytics/ - Kinesis Firehose: https://aws.amazon.com/kinesis/data-firehose/ - Must-read on Kinesis: Anahit Pogosova’s two-part blog series: https://dev.solita.fi/2020/05/28/kinesis-streams-part-1.html & https://dev.solita.fi/2020/12/21/kinesis-streams-part-2.html - Cloudonaut Kinesis vs MSK: https://cloudonaut.io/versus/messaging/kinesis-data-streams-vs-msk/ - Deep Dive on Lambda Consumers for Kinesis / Heeki Park - re:invent 2020 https://www.youtube.com/watch?v=tCYwc7-wwsU You can listen to AWS Bites wherever you get your podcasts. Find all the links on https://awsbites.com Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Luciano and Eoin deep dive into SNS discussing what it does, how it differs from EventBridge and SQS and how you can use it to send messages to customers but also for microservices communication. In this new episode dedicated to AWS events and messaging services, we learn everything there is to know about SNS including advantages, limitations and cost. This episode complements the episode about EventBridge, giving another perspective on when to use SNS and when to pick EventBridge instead. In this episode, we mentioned the following resources: - Our previous episode about EventBridge: https://www.youtube.com/watch?v=UjIE5qp-v8w - Our previous episode about all things SQS: https://www.youtube.com/watch?v=svoA-ds8-8c - Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY - A comparison between EventBridge and SNS by Cloudonaut: https://cloudonaut.io/eventbridge-vs-sns/ This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Eoin and Luciano dive into the world of EventBridge, showcasing its potential for event-driven applications. They explore various event types, including AWS and third-party events, and discuss delivery guarantees. The duo shares insights on configuring EventBridge rules and highlights its integration with Lambda and Kinesis. They also tackle pricing pitfalls and offer tips to enhance observability. With practicality in mind, they examine the integration of EventBridge and SQS for reliable message handling, ensuring listeners walk away with actionable knowledge.

Luciano and Eoin take a deep dive into SQS as part of a series on AWS event services and event-driven architecture. We talk about the kind of problems SQS can solve, all of the SQS features and how to configure and use SQS to achieve reliability and scalability without all the complexity. We also take some time to detail how SQS works with Lambda in terms of scaling, batching and filtering. In this episode, we gave a special mention to three highly-recommended re:Invent 2021 talks on the topic of Enterprise Integration Patterns with AWS services: AWS re:Invent 2021 - Application integration patterns for microservices - Gregor Hohpe - https://www.youtube.com/watch?v=ttJAIQf7cTw AWS re:Invent 2021 - Building next-gen applications with event-driven architectures - Sam Dengler - https://www.youtube.com/watch?v=U5GZNt0iMZY AWS re:Invent 2021 - AWS re:Invent 2021 - Application integration patterns for microservices - Dirk Froehner - https://www.youtube.com/watch?v=QhfuzEkN3Ck - In addition, we mentioned the following resources. - SQS: https://aws.amazon.com/sqs/ - Using Lambda with SQS: https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html - Lambda SQS Scaling: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-sqs-scaling/ - SLIC Watch (Serverless plugin for easy dashboards and alarms): https://github.com/fourTheorem/slic-watch  This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige