The New Stack Podcast

Deliveroo, a prominent food delivery company, relies on Apache Flink, a distributed processing engine, to enhance its three-sided marketplace, connecting delivery drivers, restaurants, and customers. Seeking to improve real-time data streaming and gain insights into customer behavior, Deliveroo transitioned to Flink, comparing it to alternatives like Apache Spark and Kafka Streams. Flink, with feature parity to their previous platform, offered stability and scalability. They initially experimented with Flink on Kubernetes but turned to the Amazon Managed Service for Flink (MSF) for enhanced support and maintenance.Engineers from Deliveroo, Felix Angell and Duc Anh Khu, emphasized the need for flexibility in data modeling to accommodate their fast-paced product development. However, flexibility can be complex, often requiring data model adjustments. They expressed the desire for a self-serve configuration feature in MSF, allowing easy customization of low-level settings and auto-scaling based on application metrics. This move to Flink and MSF has empowered Deliveroo to focus on core responsibilities like continuous integration and delivery while efficiently managing their data processing needs.Learn more from The New Stack about Apache Flink and AWS:Kinesis, Kafka and Amazon Managed Service for Apache FlinkApache Flink for Real Time Data AnalysisApache Flink for Unbounded Data Streams Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Over the past five to ten years, the testing of microservices has seen significant growth. This surge in testing can be attributed to the increasing adoption of microservices and Kubernetes, which signify a shift away from monolithic application architectures. Bruno Lopes, a leader at Kubernetes company incubator Kubeshop, noted this trend. Kubeshop has initiated six Kubernetes projects, including TestKube, a Kubernetes native testing framework led by Lopes.This rise in testing is making it more accessible to a wider audience and is enhancing the developer experience through automation. Developers now have more time to focus on innovation rather than manual testing. However, there is often a disconnect between development and testing, as developers move quickly, outpacing organizational adaptation to modern testing methods.Lopes emphasized the importance of testing before production deployment and advocated for creating production-resembling testing environments that allow for rapid deployment without waiting for manual tests. This approach is particularly critical for Site Reliability Engineering (SRE) teams who need to respond quickly to issues and minimize downtime for customers. In some cases, it's necessary to run tests within Kubernetes itself, a concept that may take time for companies to fully embrace as the developer experience continues to improve.Learn more from The New Stack about Kubernetes, Testing and TestKube:Testkube: A Cloud Native Testing Framework for KubernetesTop 5 Challenges in Modern Kubernetes TestingWhy You Should Start Testing in the Cloud Native Way Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Deepthi Mohan and Nagesh Honnalii from AWS discuss Apache Flink and Amazon Managed Service for Apache Flink. They highlight use cases like streaming ETL, real-time analytics, and complex event processing. They trace the origins of MSF back to Amazon Kinesis and Apache Kafka. They also discuss the advantages of using a managed service for Flink, such as ease of use, scalability, and security. The podcast also covers how to get started with Amazon Managed Service for Apache Flink, including pricing details.

Modern developer conferences like the upcoming Infobip Shift Conference in Croatia are centered around themes. At this particular event for developers, you can expect a lot of focus to be on the developer experience and artificial intelligence (AI).Ivan Burazin, Chief Development Experience Officer at InfoBip, joined us on the show and emphasizes that developers spend a substantial portion of their time not coding, often losing 50 to 70% of their productive hours to non-coding activities, such as setting up environments, running tests, and building code. This highlights the importance of improving the developer experience to enhance productivity.The developer experience has both internal and external dimensions. Externally, it impacts customer experience, while internally, it influences development velocity. A better developer experience translates to faster and more efficient coding.The Shift Conference will feature talks on six stages, one of which will focus on the developer experience, addressing its internal and external aspects. Additionally, AI will take center stage at another segment of the conference.Although there may not be an abundance of true AI experts taking the stage, the focus will be on how individuals and companies can leverage AI to create products and services. It's recognized that AI will play a pivotal role in the future of every industry, and the conference aims to explore practical applications and strategies for integrating AI into various businesses.Overall, the Shift Conference aims to address the challenges developers face in optimizing their productivity and explore the growing importance of AI in shaping the future of businesses and products.Learn more from The New Stack about the developer experience and InfoBip Shift:7 Principles and 10 Tactics to Make You a 10x DeveloperThe Challenges of Marketing Software Tools to DevelopersA Guide to Better Developer Experience Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

This episode delves into Apache Flink, a versatile platform for executing both batch and real-time streaming data analysis tasks. This session marks the beginning of a three-part series unveiling Amazon Web Services' (AWS) new managed service built on Flink. Future episodes will explore this service in detail and examine customer experiences.The podcast features insights from Danny Cranmer, a principal engineer at AWS and an Apache Flink PMC and Committer, along with Hong Teoh, a software development engineer at AWS.Flink stands out as a high-level framework for defining data analytics jobs, accommodating both batch and streaming data sets. It offers APIs for building analysis jobs in various languages, including Java, Python, and SQL. Flink also provides a distributed job execution engine with fault tolerance and horizontal scaling capabilities.One prominent use case is Extract-Transform-Load (ETL), where raw data is swiftly processed for specific workloads. Flink excels in delivering low-latency transformations for unbounded data streams. Additionally, Flink supports event-driven applications, responding immediately to triggers such as user requests for weather data.Flink ensures exactly-once processing, critical for scenarios like financial transactions. It employs checkpoints to maintain data integrity in case of node failures.The podcast also touches on AWS's role in supporting the open-source Flink project and the future outlook for this powerful data processing framework.Learn more from The New Stack about Apache Flink:3 Reasons Why You Need Apache Flink for Stream ProcessingApache Flink for Unbounded Data Streams8 Real-Time Data Best Practices Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Renowned technologist Adrian Cockcroft discusses the process of fine-tuning Large Language Models (LLMs) through prompt engineering. Crafting specific prompts guides AI's output. Fine-tuning of LLMs using internal data like wiki pages enables better domain and process understanding. Vector databases are in demand for enhanced information retrieval from LLMs. Cockcroft explores the challenges and potential pitfalls of code-generating assistants. He also highlights the importance of embracing new technologies and startups in the fast-moving tech field.

TechWorld with Nana is one of the most popular resources for people looking to get into or progress a DevOps career. Nana Janashia, the creator of TechWorld with Nana, is a DevOps trainer and consultant who joined us to discuss why DevOps is needed now more than ever and how this is the perfect time to begin a career in DevOps.Host Alex Williams and Nana go over the key concepts of DevOps. Then they talk about how the complexity of tools can sidetrack and complicate the learning process for those new to DevOps and why focusing on concepts rather than tools the way to go. Before wrapping up the conversation, they even talk about the best ways for people to get involved who are new to DevOps.Nana's journey into DevOps commenced during her time as an engineer in Austria, where she began exploring Kubernetes. As inquiries from colleagues poured in, she recognized her knack for demystifying complex topics, catalyzing her passion for teaching. Viewers attest to switching to DevOps careers after watching her videos.Throughout the conversation, we learned how people can discover the world of DevOps through TechWorld with Nana as an expert guide. With a large YouTube audience, online courses, workshops, and corporate training, Nana has empowered countless individuals in advancing their DevOps expertise. The six-month boot camps from TechWorld with Nana encompass a comprehensive curriculum, starting with fundamentals and culminating in hands-on programming abilities, Python automation, configuration management, and Prometheus-based monitoring.Nana underscores that DevOps, still a relatively nascent profession, suffers from role ambiguity both among engineers and within companies aspiring to implement it. This confusion stems from differing workflows and environments when engineers switch jobs. Nana's insights bring clarity to these challenges, acknowledging the evolving chaos of the DevOps culture and its driving force for innovation in managing intricate distributed technologies.Learn more about DevOps from TNS, Roadmap (our sister site), and TechWorld with Nana:TechWorld with Nana - DevOps  BootcampTechWorld with Nana - DevSecOps BootcampDevOps Learning RoadmapDevOps News, Trends, and Analysis Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Amanda Brock, CEO of OpenUK, discusses the challenges of labeling AI as open source amid legal ambiguities. The podcast explores the evolution of open source licensing and the complications of applying it to AI. The discussion includes the contentious 'Llama 2' example and the need for a new definition of open source for AI. The podcast also covers the transition to Server Side Public License and the role of regulators in Open Source AI.

Discover how large language models and generative AI are revolutionizing DevOps with PromptOps. The company, initially known as CtrlStack, introduces its unique process engine that comprehends human requests, reads knowledge bases, and generates code on the fly to accomplish tasks. Dev Nag, the CEO, explains how PromptOps saves users time and money by automating routine operations in  this podcast episode with The New Stack.Dev Nag is joined by GK Brar, PromptOps' founding engineer, and our host Joab Jackson as they delve into the concept of generative AI and its potential benefits for DevOps. Traditionally, DevOps tasks often involve repetitive troubleshooting and reporting, making automation essential. PromptOps specializes in intent matching, understanding nuanced requests and providing the right solutions.Notably, PromptOps employs generative AI offline to prepare for automating common actions and enhancing the user experience. Unlike others, PromptOps aims beyond simple enhancements. It aspires to transform the entire DevOps landscape by leveraging this groundbreaking technology.Tune in to the podcast to gain deeper insights into this transformative approach that PromptOps brings to DevOps thanks to the power and possibilities of generative AI.Learn more from The New Stack about DevOps and PromptOps:DevOps News, Trends, Analysis and ResourcesHow to Use ChatGPT for IT Security AuditWhat We Learned from Building a Chatbot Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

In this episode, Matt Butcher, CEO of Fermyon Technologies, discusses the potential impact of the component model on WebAssembly (Wasm) and its integration into the cloud-native landscape. WebAssembly is a binary instruction format enabling code to run anywhere, written in developers' preferred languages. The component model aims to provide a common way for WebAssembly libraries to express their needs and connect with other modules, reducing the barriers and maintenance of existing libraries. Butcher believes this model could be a game changer, allowing new languages to compile WebAssembly and utilize existing libraries seamlessly.WebAssembly also shows promise in delivering on the long-awaited potential of serverless computing. Unlike traditional virtual machines and containers, WebAssembly boasts a rapid startup time and addresses various developer challenges. Butcher states that developers have been eagerly waiting for a platform with these characteristics, hinting at a potential resurgence of serverless. He clarifies that WebAssembly is not a "Kubernetes killer" but can coexist with container technologies, evident from the Kubernetes ecosystem's interest in supporting WebAssembly.The episode explores further developments in WebAssembly and its potential to play a central role in the cloud-native ecosystem.Learn more from The New Stack about WebAssembly and Fermyon Technologies:WebAssembly Overview, News, and TrendsWebAssembly vs. KubernetesFermyon Cloud: Save Your WebAssembly Serverless Data Locally Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.