The New Stack Podcast

Fatima Sarah Khalid, keynote speaker at the Linux Foundation's Open Source Summit Europe, discusses the importance of allyship in open source communities. She highlights improving diversity and inclusion, making open source events more inclusive, and GitLab's unique approach to allyship. Khalid shares commitments to embrace allyship, such as educating oneself and centering underrepresented voices.

The podcast discusses the potential impact of the Cyber Resilience Act on open source development in Europe. It explores the risks and challenges faced by individuals and organizations, the importance of forks, and the need for better communication and advocacy for open source. It also highlights the importance of opposing a decision that could hinder open source development in Europe.

Uma Daniel discusses the complexities in the global economy, the concept of FinOps, debunking misconceptions, implementing FinOps by creating awareness and building a team, and USD's approach to FinOps and customer engagement.

Madhukar Kumar, CMO of SingleStore, discusses experiments with large language models (LLMs), including Gorilla. He explains 'retrieval augmented generation' (RAG) to keep LLMs updated and addresses the limitation of LLMs being 'frozen in time.' Kumar highlights SingleStore's use of RAG to provide current and contextually accurate information, making AI applications more reliable and responsive for enterprises.

Cloud-native observability in multi-cloud environments is increasingly complex, creating challenges of rising incidents and costly tools. The podcast explores the need to prioritize alerts and shift security responsibility to developers. It also dives into the cultural shift towards DevOps and the gap in addressing developer needs with current observability tools. The guest suggests empowering developers to run and maintain their software in production with cloud-native observability tools, providing insights into complex environments. The podcast also covers the impact of multi-cloud and cloud-native architecture, open source updates, and trends in observability.

Platform engineering is gaining prominence due to the need for faster application deployment, which directly impacts business velocity. Valentina Alaria, Senior Director of Product at VMware, emphasizes that not all organizations pursuing platform engineering have the same goals, context, or pain points. They tailor solutions to each organization's specific needs. Some focus on rapid onboarding for junior developers, while others aim to reduce complexity, friction, and support larger development teams with fewer operational staff.Platform engineering aims to streamline collaboration between developers and operations engineers. Developers want portable code and the ability to focus on coding without worrying about production requirements. Operations engineers and platform teams seek a seamless environment for deploying applications in different contexts.Successful platform engineering initiatives involve strong collaboration models, fostering a cooperative approach rather than a siloed one. The goal is to create applications and value for the organization by facilitating effective interaction between developers and operations engineers.This podcast episode, hosted by Alex Williams of TNS, also delves into VMware Tanzu's latest tools for supporting platform engineering.Learn more from The New Stack about platform engineering and VMware Tanzu:Platform Engineering Overview, News and Trends6 Patterns for Platform Engineering SuccessA Guide to Open Source Platform EngineeringStreamline Platform Engineering with Kubernetes

ByConity is an open source project that emerged from ByteDance's use of Clickhouse, an open-source database system, to address their growing data volume. ByConity focuses on enhancing the separation of compute and storage, improving multitenancy support, and optimizing query performance in cloud-native environments.ByteDance's Vini Jaiswal, a principle developer advocate at the parent company of TikTok, highlights the power of open source in fostering innovation and collaboration. She shares her personal experience of leveraging open source to solve problems quickly and efficiently. She emphasizes the importance of getting involved in open source, even for those who might be hesitant, and suggests starting by identifying a pain point and making small contributions.ByConity's architecture, which separates compute and storage, offers benefits like preventing data lake corruption, read and write separation, elasticity, and scalability. Jaiswal also mentions her previous experience with open source during her time at CitiBank, where she realized how open source accelerated digital transformations.Throughout the conversation, Jaiswal underscores the strength of open source communities in collectively addressing challenges. She encourages listeners to embrace open source and start contributing, emphasizing how even small contributions can lead to significant impacts over time.The episode also delves into Jaiswal's involvement with other open source projects, such as PyTorch, and explores the intersection of open source and generative AI.Learn more from The New Stack about open source and cloud native environments:What Is 'Cloud Native' (and Why Does It Matter)?Cloud Native Ecosystem News and ResourcesHow to Build an Open Source Community

Along with discussing the emergence and ascension of platform engineering in this episode, we also discuss the role that Humanitec plays in helping organizations establish platforms for developers, as well as Backstage, a popular open source internal developer platform that was developed by Spotify for its own developers.An IDP, our guest Kaspar Von Grünberg explained, is a standardized interface for developers to build applications using a  golden path of vetted tools and libraries, allowing for a high degree of efficiency for both the developers themselves as well as the engineers who are supporting the developers. They can include an integration and delivery plane, a continuous integration registry, a platform orchestrator, observability tools and a resource plane."How you're consuming this is a little bit up to the individual preference of the user, and what the platform team has configured for you. So we're seeing some teams like to use a user interface and some teams like to use code based interactions," Von Grünberg explained.In some ways, a IDP is reminiscent of the platform-as-a-service packages of a decade ago. They also were designed to help developer efficiency, though devs chafed at the limited number of tools they were allowed to use  in these walled gardens. That was a mistake, Von Grünberg said.Those platforms required developers to use a small set of pre-defined times."We don't want to get back to those times, which is why we want to provide sensible defaults," Von Grünberg said. A good IDP will provide developers with "golden paths" or "paved roads" as Netflix calls them."Developers can stay on those paths if they want," Von Grünberg said. They can enjoy the security default and service-level agreements (SLAs) from the engineers. But developers are also free to leave the path and make low-level configurations on their own as well."Good platform engineering is never about covering all the use cases," he said.Learn more from The New Stack about platform engineering and Humanitec:Platform Engineering Overview, News, and TrendsHow to Pave Golden Paths That Actually Go SomewhereBuild Your IDP at Light Speed with a Platform Reference Architecture

John Willis, a technologist and author, cautions against blindly following vendor recommendations for AI products. He suggests a "DevOps redo" to encourage experimentation and collaboration. Willis highlights the importance of educating teams on data management techniques, including retrieval augmentation. He emphasizes the need for data cleansing to prevent undesirable code or sensitive information. The podcast also discusses the rise of Shadow AI, the difference between co-completion tools and proprietary tools, and the challenges of responsible AI implementation.

Deliveroo, a prominent food delivery company, relies on Apache Flink, a distributed processing engine, to enhance its three-sided marketplace, connecting delivery drivers, restaurants, and customers. Seeking to improve real-time data streaming and gain insights into customer behavior, Deliveroo transitioned to Flink, comparing it to alternatives like Apache Spark and Kafka Streams. Flink, with feature parity to their previous platform, offered stability and scalability. They initially experimented with Flink on Kubernetes but turned to the Amazon Managed Service for Flink (MSF) for enhanced support and maintenance.Engineers from Deliveroo, Felix Angell and Duc Anh Khu, emphasized the need for flexibility in data modeling to accommodate their fast-paced product development. However, flexibility can be complex, often requiring data model adjustments. They expressed the desire for a self-serve configuration feature in MSF, allowing easy customization of low-level settings and auto-scaling based on application metrics. This move to Flink and MSF has empowered Deliveroo to focus on core responsibilities like continuous integration and delivery while efficiently managing their data processing needs.Learn more from The New Stack about Apache Flink and AWS:Kinesis, Kafka and Amazon Managed Service for Apache FlinkApache Flink for Real Time Data AnalysisApache Flink for Unbounded Data Streams