#005 - Kubernetes For Humans Podcast with Gwen Shapira (Nile)
Sep 20, 2023
auto_awesome
Gwen Shapira discusses transitioning open-source projects to SaaS offerings with Kubernetes. She shares insights on managing databases for major organizations and the challenges of running stateful services on Kubernetes. The conversation covers her software journey, platform engineering expertise, early experiences with Kubernetes at Confluent, managing stateful workloads, and the future impact of Kubernetes in the tech industry.
Transitioning Apache Kafka to SaaS with Kubernetes poses architectural challenges.
Kubernetes integration reshapes platform engineering, emphasizing scalability and alignment with future solutions.
Deep dives
Gwen Shafira's Introduction to Kubernetes World
Gwen Shafira, co-founder of a software-as-a-service platform, shares insights on transitioning Apache Gafga into a SaaS product, highlighting the challenges faced with Kubernetes involvement. Her background in managing large databases and data centers shaped her platform engineering expertise, emphasizing the architectural aspects and scalability considerations within software development.
Early Adoption of Kubernetes at Confluent
Gwen discusses joining Confluent at an early stage, where the move to a cloud solution was imperative. The integration of Kubernetes for hosting Kafka services challenged traditional perceptions of running stateful services, leading to strategic decisions in aligning Kafka's behavior with Kubernetes principles. The evolution towards Kubernetes in the company's offerings showcased a commitment to future-oriented solutions.
Platform Engineering Growth and Balancing Stateful Workloads
The growth of the platform team at Confluent highlights the essential role of platform engineers in enabling application teams. Challenges in managing stateful workloads on Kubernetes revealed the importance of aligning platform expectations, with Kubernetes' self-healing capabilities influencing a shift towards trusting platforms for operational management. Insights on running stateful workloads in Kubernetes underscore the industry's evolving approach to managing databases within cloud-native environments.
Gwen Shapira is the Co-Founding CPO at Nile. Previously a system architect at Confluent (employee no. 12!) helping customers achieve success with their Apache Kafka implementation. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. She specializes in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, an author of "Hadoop Application Architectures", and a frequent presenter at data-driven conferences. Gwen is also a committer on the Apache Kafka and Apache Sqoop projects.
Buy Gwen's books on Amazon: https://www.amazon.com/Kafka-Definitive-Real-Time-Stream-Processing/dp/1492043087/ref=sr_1_1?crid=35PP0X76U285K&keywords=Kafka%3A+The+Definitive+Guide%3A+Real-Time+Data+and+Stream&qid=1695222358&s=digital-text&sprefix=%2Cdigital-text%2C318&sr=1-1
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode