AWS Bites

In this episode, we discuss using AWS Lambda for machine learning inference. We cover the tradeoffs between GPUs and CPUs for ML, tools like ggml and llama.cpp for running models on CPUs, and share examples where we've experimented with Lambda for ML like podcast transcription, medical imaging, and natural language processing. While Lambda ML is still quite experimental, it can be a viable option for certain use cases. 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources. Episode "46. How do you do machine learning on AWS?": https://awsbites.com/46-how-do-you-do-machine-learning-on-aws/ Episode "108. How to Solve Lambda Python Cold Starts": https://awsbites.com/108-how-to-solve-lambda-python-cold-starts/ ggml (the framework): https://github.com/ggerganov/ggml ggml (the company): https://ggml.ai llama.cpp: https://github.com/ggerganov/llama.cpp whisper.cpp: https://github.com/ggerganov/whisper.cpp whisper.cpp WebAssembly demo: https://whisper.ggerganov.com/ ONNX Runtime: https://onnxruntime.ai/ An example of using whisper.cpp with the Rust bindings: https://github.com/lmammino/whisper-rs-example Project running Whisper.cpp in a Lambda function: https://github.com/eoinsha/whisper_lambda_cpp AWS Lambda Image Container Chest X-Ray Example: https://github.com/fourTheorem/lambda-image-cxr-detection Episode "103. Building GenAI Features with Bedrock": https://awsbites.com/103-building-genai-features-with-bedrock/⁠ Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - ⁠⁠⁠⁠https://twitter.com/eoins⁠⁠⁠⁠ - ⁠⁠⁠⁠https://twitter.com/loige⁠⁠

This episode of the AWS Bites Podcast provides an overview of the AWS Project Development Kit (PDK), an open-source tool to help bootstrap and maintain cloud projects. We discuss what PDK is, how it can help generate boilerplate code and infrastructure, keep configuration consistent across projects, and some pros and cons of using a tool like this versus doing it manually. Is PDK something you should use for your cloud projects? Let's find out! 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources. The official PDK website (and documentation): https://aws.github.io/aws-pdk/ Our previous episode "16. What are the pros and cons of CDK?": https://awsbites.com/16-what-are-the-pros-and-cons-of-cdk/ Our previous episode "93. CDK Patterns - The Good, The Bad and The Ugly": https://awsbites.com/93-cdk-patterns-the-good-the-bad-and-the-ugly/ Projen's official website: https://projen.io/ Introduction talk to Projen at CDK Day 2020: https://www.youtube.com/watch?v=SOWMPzXtTCw Our previous episode "70. How do you create good AWS diagrams?": https://awsbites.com/70-how-do-you-create-good-aws-diagrams/ Building a shopping list app with PDK (tutorial): https://aws.github.io/aws-pdk/getting_started/shopping_list_app.html PDK in-depth developer guides: https://aws.github.io/aws-pdk/developer_guides/index.html Opinion by Vlad Ionescu on X: https://twitter.com/iamvlaaaaaaad/status/1743608823896592640 Yeoman: https://yeoman.io/ CookieCutter: https://github.com/cookiecutter/cookiecutter Terraform project generation example: https://github.com/conzy/terraform-demo Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - ⁠⁠⁠https://twitter.com/eoins⁠⁠⁠ - ⁠⁠⁠https://twitter.com/loige⁠⁠

This podcast discusses using Python for data science on AWS Lambda. They cover the pros and cons of Lambda for data science workloads, benchmark cold start times, and compare different deployment options. Container images provide faster cold starts once caches are warmed up. Overall, Lambda is a good fit for bursty and high concurrency data science workloads.

Episode covers expert opinions on AWS announcements from re:Invent 2023, advice for starting cloud journey, predictions for serverless future, multi-cloud considerations, and AI impact on developers. Speakers discuss gaining practical knowledge, building projects, open source contributions, favorite services, challenges in serverless architecture, and the role of AI in decision-making.

Luciano and Eoin chat about Luciano's experience attending AWS re:Invent 2023 in Las Vegas for the first time. They talk about the massive scale of the event, logistical challenges getting around between venues, highlights from the keynotes and announcements, and tips for networking and getting the most out of re:Invent. Luciano shares his perspective on the AI focus, meeting people in real life after connecting online, rookie mistakes to avoid, and why re:Invent is worth the investment for anyone working in the AWS space. AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources. - Amazon Q: https://aws.amazon.com/blogs/aws/introducing-amazon-q-a-new-generative-ai-powered-assistant-preview/ - Efi Merdler-Kravitz's talk on "Rustifying serverless" with AWS Lambda (YouTube): https://www.youtube.com/watch?v=Mdh_2PXe9i8 - ElastiCache Serverless for Redis and Memcached: https://aws.amazon.com/blogs/aws/amazon-elasticache-serverless-for-redis-and-memcached-now-generally-available/ - Throughput increase and dead letter queue redrive for SQS FIFO: https://aws.amazon.com/blogs/aws/announcing-throughput-increase-and-dead-letter-queue-redrive-support-for-amazon-sqs-fifo-queues/ - Step Functions Workflow Studio in AWS Application Composer: https://aws.amazon.com/blogs/aws/aws-step-functions-workflow-studio-is-now-available-in-aws-application-composer/ - Lambda scales 12x faster: https://aws.amazon.com/blogs/aws/aws-lambda-functions-now-scale-12-times-faster-when-handling-high-volume-requests/ - Step Function redrive from a failed state: https://aws.amazon.com/blogs/compute/introducing-aws-step-functions-redrive-a-new-way-to-restart-workflows/ Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - https://twitter.com/eoins - https://twitter.com/loige #aws #reinvent2023 #reinvent #networkingevents

Dive into the world of integration testing for event-driven systems on AWS. The discussion explores the challenges of testing events and introduces the new Integration Application Test Kit (IATK). Learn about effective strategies like logging and end-to-end testing while navigating through the promising yet imperfect IATK features. Discover the benefits of AWS X-Ray for improved accuracy and the potential of an open-source testing tool to enhance AWS service integration. It's a whirlwind of insights perfect for developers!

In this episode, we celebrate AWS Lambda's 9th birthday by taking a deep dive into Lambda runtimes. We discuss how Lambda works, compare official runtimes vs. custom runtimes, and explain when and why building a custom runtime might be worth the effort. We talk through how custom runtimes work, options for deploying them, and potential use cases where they could be beneficial over standard runtimes. 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at ⁠⁠⁠fourtheorem.com⁠⁠⁠! In this episode, we mentioned the following resources: The original announcement launching AWS Lambda in 2014: https://aws.amazon.com/about-aws/whats-new/2014/11/13/introducing-aws-lambda/ List of official Lambda runtimes: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html Library implementing the Rust runtime: https://github.com/awslabs/aws-lambda-rust-runtime Library implementing the Go runtime: https://github.com/aws/aws-lambda-go/ Library implementing the C++ runtime: https://github.com/awslabs/aws-lambda-cpp Third-party PHP runtime (Bref): https://bref.sh/ Third-party Swift runtime: https://github.com/swift-server/swift-aws-lambda-runtime Official documentation on how to build a custom runtime: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html An official tutorial on how to build a custom runtime: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-walkthrough.html List of all the environment variables available to a runtime: https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html#configuration-envvars-runtime Lambda Response Streaming official docs: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html#runtimes-custom-response-streaming Our previous episode on Lambda Response Streaming: https://awsbites.com/77-how-do-you-use-lambda-response-streaming/ On-demand Container Loading in AWS Lambda (paper): https://arxiv.org/abs/2305.13162 Deep dive on container support for AWS Lambda (Eoin's article): https://dev.to/eoinsha/container-image-support-in-aws-lambda-deep-dive-2keh When is the Lambda Init Phase Free, and when is it Billed? (article by Luc van Donkersgoed): https://lucvandonkersgoed.com/2022/04/09/when-is-the-lambda-init-phase-free-and-when-is-it-billed/⁠ You can listen to AWS Bites wherever you get your podcasts: Apple Podcasts:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://podcasts.apple.com/us/podcast/aws-bites/id1585489017⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Spotify: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Google: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Breaker:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://www.breaker.audio/aws-bites⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ RSS:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://anchor.fm/s/6a3312a0/podcast/rss

In this episode, we discuss how we automated generating YouTube descriptions, chapters and tags for our podcast using Amazon's new GenAI tool: Bedrock. We provide an overview of Bedrock's features and how we built an integration to summarize podcast transcripts and extract relevant metadata using the Anthropic Claude model. We share the prompt engineering required to instruct the AI, and details on our serverless architecture using Step Functions, Lambda, and EventBridge. We also discussed Bedrock pricing models and how we built a real-time cost-monitoring dashboard. Overall, this automation saves us substantial manual effort while keeping costs low. We hope this episode inspires others to explore building their AI workflows with Bedrock. 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at ⁠⁠fourtheorem.com⁠⁠! In this episode, we mentioned the following resources: The GitHub repository with the code for the AWS Bites website: https://github.com/awsbites/aws-bites-site Episoder: the open source project we created to handle this Bedrock-based automation: https://github.com/fourTheorem/episoder Podwhisperer: out other open source automation that creates accurate transcripts for our podcast: https://github.com/fourTheorem/podwhisperer/ Amazon Bedrock Workshop: https://catalog.us-east-1.prod.workshops.aws/workshops/a4bdb007-5600-4368-81c5-ff5b4154f518/en-US Using generative AI on AWS for diverse content types (workshop): https://catalog.workshops.aws/genai-on-aws/en-US/01-introduction Deploying a Multi-Model and Multi-RAG Powered Chatbot Using AWS CDK on AWS: https://github.com/aws-samples/aws-genai-llm-chatbot/tree/main You can listen to AWS Bites wherever you get your podcasts: Apple Podcasts:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://podcasts.apple.com/us/podcast/aws-bites/id1585489017⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Spotify: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Google: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Breaker:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://www.breaker.audio/aws-bites⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ RSS:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://anchor.fm/s/6a3312a0/podcast/rss

In this episode, we have the pleasure of speaking with Jeremy Daly, CEO of Ampt and a leader in the AWS and serverless community. We discuss Jeremy's journey into AWS and serverless, the prolific open source work and content he creates, the evolution of serverless over the years, common myths about serverless, and, finally, the story behind building Ampt to improve the developer experience. Jeremy provides his perspective on the state of serverless and predictions for the future and it also gives some fantastic pieces of advice for wannabe tech-entrepreneurs! 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at ⁠fourtheorem.com⁠! In this episode, we mentioned the following resources: Serverless Microservice patterns for AWS (article by Jeremy): https://www.jeremydaly.com/serverless-microservice-patterns-for-aws/ Datadog research: The state of Serverless: https://www.datadoghq.com/state-of-serverless/ Bref: serverless runtime for PHP: https://bref.sh/ Ampt original announcement blog post: https://www.serverless.com/blog/serverless-cloud-spins-off-as-ampt Our previous episode on Ampt: https://awsbites.com/100-exploring-ampt-a-new-way-to-build-cloud-apps-on-aws/ Long-running tasks and smart compute with Ampt: https://getampt.com/blog/introducing-long-running-tasks/ Ampt official website: https://getampt.com/ Jeremy's official website: https://www.jeremydaly.com/ Jeremy on X (ergo Twitter): https://twitter.com/jeremy_daly Jeremy's newsletter "Off-by-none": https://offbynone.io/ You can listen to AWS Bites wherever you get your podcasts: Apple Podcasts:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://podcasts.apple.com/us/podcast/aws-bites/id1585489017⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Spotify: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Google: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Breaker:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://www.breaker.audio/aws-bites⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ RSS:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://anchor.fm/s/6a3312a0/podcast/rss

Today we embark on a fascinating journey into the world of AWS Lambda functions and how to make them accessible to the public. In a recent use case, involving the creation of a public Lambda function for AWS users, we asked ourselves some interesting questions. How can you securely, cost-effectively, and conveniently publish AWS resources, especially Lambda functions, for others to use? And... can we possibly make some money out of this? Join us as we explore various options and share our findings for making your AWS resources available to the world. We dive into the Serverless Application Repository (SAR), an AWS treasure trove for publishing resources. And SAR isn't the only way! We also discuss alternatives like CloudFormation templates, GitHub publishing, Terraform modules, and container images. We explore the pros and cons of these methods and debate the implications in terms of cost, security, and ease of use. Finally, we touch on the AWS Marketplace as a platform to monetize your AWS resources. 💰 SPONSORS 💰 AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com! In this episode, we mentioned the following resources: Our previous episode about HTTP Lambdas (including an overview of Rest vs HTTP API Gateways): https://awsbites.com/74-function-urls-vs-api-gateway/ Example of publicly accessible Lambda code using CloudFormation templates on GitHub: https://github.com/eoinsha/public-s3-lambda-code-example AWS SAR (Serverless Application Repository): https://aws.amazon.com/serverless/serverlessrepo/ The Terraform Registry: https://registry.terraform.io/ AWS Marketplace: https://aws.amazon.com/marketplace "Selling an AMI and a CloudFormation template as an alternative to SaaS" by Cloudonaut: https://cloudonaut.io/selling-ami-cloudformation-alternative-saas/ You can listen to AWS Bites wherever you get your podcasts: Apple Podcasts:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://podcasts.apple.com/us/podcast/aws-bites/id1585489017⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Spotify: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Google: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Breaker:⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ https://www.breaker.audio/aws-bites⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ RSS:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://anchor.fm/s/6a3312a0/podcast/rss