Dive into innovative engineering as Alex discusses Red Panda's unique architecture, setting it apart from traditional messaging systems like Kafka. Unravel the complexities of optimizing memory management and latency for high-performance streaming. Explore the benefits of the 'thread per core' design for improved concurrency and reduced latency. Discover the importance of storage protocol correctness and the rigor of formal verification methods. This conversation highlights a future where streamlined data processing meets cutting-edge technology.

The GeekNarrator memberships can be joined here: https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/joinMembership will get you access to member only videos, exclusive notes and monthly 1:1 with me. Here you can see all the member only videos: https://www.youtube.com/playlist?list=UUMO_mGuY4g0mggeUGM6V1osdA------------------------------------------------------------------------------------------------------------------------------------------------------------------About this episode: ------------------------------------------------------------------------------------------------------------------------------------------------------------------In this episode, we talk to Søren Schmidt, Co-Founder and CEO of Prisma, discussing the evolution of Prisma from a backend as a service to a popular ORM and now to Prisma Postgres. He shares insights into the challenges faced during this journey, the importance of user feedback, and the innovative architecture of Prisma Postgres, which leverages micro VMs for performance optimization. The conversation also touches on the complexities of managing data centers and the strategies employed to ensure a seamless user experience. In this conversation, Søren Schmidt discusses the details about Postgres snapshots, their impact on performance, and the mechanisms for fault tolerance. He explains how Pulse change data capture works and how Prisma Postgres simplifies database management for users. Chapters00:00 Introduction to Prisma and Its Evolution03:00 The Journey from ORM to Prisma Postgres06:00 Simplifying Database Management09:01 Understanding Prisma Postgres Architecture12:12 The Role of Accelerate in Query Routing14:51 Optimizing Query Processing with Micro VMs18:12 Maintaining Postgres Integrity in a Micro VM Environment21:07 User Experience and Community Feedback23:57 Challenges of Data Center Management27:09 Cold Starts and Performance Optimization34:30 Understanding Snapshots in Postgres38:55 Snapshot Mechanisms and Fault Tolerance44:09 Change Data Capture with Pulse55:07 Transitioning to Prisma Postgres58:45 Community and Getting Started with Prisma PostgresSome blogs worth checking out:https://www.prisma.io/blog/prisma-postgres-the-future-of-serverless-databaseshttps://www.prisma.io/blog/cloudflare-unikernels-and-bare-metal-life-of-a-prisma-postgres-queryhttps://www.prisma.io/blog/announcing-prisma-postgres-early-accessPrisma Postgres relies heavily on the Unikraft project. There is a good introductory talk here: https://www.youtube.com/watch?v=n4wOyAuNhl0And some very technical papers here: https://unikraft.org/community/papersThe best way to get started with Prisma Postgres is to go straight to https://www.prisma.io/ ------------------------------------------------------------------------------------------------------------------------------------------------------------------Like building real stuff?------------------------------------------------------------------------------------------------------------------------------------------------------------------Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.https://app.codecrafters.io/join?via=geeknarrator------------Database internals series: https://youtu.be/yV_Zp0Mi3xsPopular playlists:Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_dModern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN

The GeekNarrator memberships can be joined here: https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/joinMembership will get you access to member only videos, exclusive notes and monthly 1:1 with me. Here you can see all the member only videos: https://www.youtube.com/playlist?list=UUMO_mGuY4g0mggeUGM6V1osdA------------------------------------------------------------------------------------------------------------------------------------------------------------------About this episode: ------------------------------------------------------------------------------------------------------------------------------------------------------------------In this episode, Kaivalya Apte and Frederic Branczyk talk about observability, focusing on continuous profiling and the role of eBPF. They discuss the evolution of profiling techniques, the importance of systematic data collection, and the challenges faced in maintaining low overhead while gathering detailed performance metrics.Frederic shares insights from his extensive experience with Prometheus and Kubernetes, emphasizing the transformative impact of continuous profiling on software performance optimization. This conversation delves into the intricacies of eBPF (Extended Berkeley Packet Filter) and its applications in profiling and performance analysis. The discussion covers the capabilities of eBPF in extending the kernel safely, the mechanisms of user space profiling, and the handling of process terminations. It also explores memory and network profiling techniques, the challenges of profiling in different programming environments, and the limitations of eBPF in certain use cases. The conversation concludes with valuable resources for those interested in learning more about eBPF and profiling techniques.Chapters:00:00 Introduction to Observability and Profiling01:17 Frederic's Background and Expertise02:11 The Importance of Continuous Profiling06:46 The Value of Continuous Profiling11:20 Understanding Profiling Data19:09 Data Structures and Performance in Profiling32:35 The Role of eBPF in Profiling42:48 Introduction to eBPF and Its Capabilities48:32 User Space Profiling and Memory Management51:39 Handling Process Termination and Agent Recovery55:27 Memory and Network Profiling Techniques01:01:33 Profiling in Different Programming Environments01:11:47 Use Cases and Limitations of eBPF in Profiling01:13:54 Resources for Learning eBPF and Profiling Techniques------------------------------------------------------------------------------------------------------------------------------------------------------------------Like building real stuff?------------------------------------------------------------------------------------------------------------------------------------------------------------------Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.https://app.codecrafters.io/join?via=geeknarrator------------------------------------------------------------------------------------------------------------------------------------------------------------------Link to other playlists. LIKE, SHARE and SUBSCRIBE------------------------------------------------------------------------------------------------------------------------------------------------------------------Database internals series: https://youtu.be/yV_Zp0Mi3xsPopular playlists:Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_dModern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsNStay Curios! Keep Learning!

Unmesh Joshi, a Principal Consultant at ThoughtWorks and author of "Patterns of Distributed Systems," dives deep into the intricacies of distributed systems. He discusses the critical role of context in applying patterns and the necessity of reading code for better implementation. Unmesh highlights common developer pitfalls and the importance of understanding consensus mechanisms, using the Paxos algorithm as an example. He also emphasizes the evolution of cloud computing practices and the human aspect of software design, urging a user-focused iterative approach.

In this discussion, Marc Brooker, a distinguished engineer at AWS, shares insights from his extensive career transforming cloud computing. He dives into the creation of Aurora Distributed SQL, emphasizing its resilience and serverless nature. The choice of PostgreSQL as a foundation is highlighted, focusing on its robust features. Marc also explains the complexities of managing distributed databases, including concurrency control and the architecture's fault tolerance. Innovative elements like Firecracker VMs are discussed for their role in enhancing scalability and performance.

Discover the game-changing integration of DuckDB with PostgreSQL, enhancing analytics capabilities. Learn how pg_duckdb addresses challenges in transaction and analytics management. Explore the intricacies of schema migration and the importance of careful upgrades. Hear insights about community contributions and how newcomers can participate in this dynamic project. Exciting discussions about optimizing query performance and the future roadmap of DuckDB and Postgres await!

Dive into the world of fault-tolerant applications with insights from the co-founders of DBOS. They tackle the complexities of building reliable systems and the architecture behind their innovative library. Explore strategies for managing workflows and side effects, like email delivery, without blocking processes. Learn about maximizing efficiency with unique serverless pricing models and the integration of AI agents. This engaging discussion highlights the importance of observability, version control, and resource management for resilient cloud solutions.

Deep Dive into Databases with Peter Zaitsev | The GeekNarrator Podcast Join host Kaivalya Apte and special guest Peter Zaitsev from Percona on this episode of the Geeknerder Podcast. They discuss Peter's fascinating journey into the world of databases, founding Percona, and the evolution of open source database solutions. Topics include the rise of PostgreSQL, the comparison between MySQL and PostgreSQL, database observability, the impact of cloud and Kubernetes on database management, licensing changes in popular databases like Redis, and career advice for database administrators and developers. Stay tuned for insights on the future of databases, observability strategies, and the role of AI in database management. 00:00 Introduction and Guest Welcome 00:14 Peter's Journey into Databases 04:15 The Rise of PostgreSQL vs MySQL 18:17 Challenges in Managing Database Clusters 24:36 Common Developer Mistakes with Databases 30:59 MongoDB's Success and Future 34:53 Redis and Licensing Changes 37:07 Elastic's License Change and Its Impact 38:25 Redis Fork and Industry Collaboration 40:27 Kubernetes and Cloud-Native Databases 47:47 Challenges in Database Upgrades and Migrations 54:58 Load Testing and Observability 01:09:02 Future of Database Administration and Development 01:15:13 Conclusion and Final Thoughts Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me. Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription. https://app.codecrafters.io/join?via=geeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

In this talk, Simon Hørup Eskildsen, an experienced software engineer and founder of Turbopuffer, shares insights from his vast experience in database scalability and object storage solutions. He dives into the challenges of building databases on object storage, discussing write-ahead logs, multi-tenancy issues, and the intricacies of handling multiple writers. Simon elaborates on trade-offs in write operations and the complexities of optimizing database read paths, offering practical strategies for performance enhancement. A must-listen for database enthusiasts!

Welcome to The GeekNarrator podcast! In this episode, host Kaivalya Apte goes deeper into the practical applications of formal methods with Jack Vanlightly, a principal technologist at Confluent. With years of experience in distributed systems, Jack discusses his journey and how formal methods have been instrumental in system design verification and bug detection. The conversation covers Jack's background, his process of using formal methods, the significance of modelling, verification, documentation, and systems learning, as well as the future evolution of tooling and its applications. Tune in to understand the intricacies of how formal methods can transform your approach to distributed systems! Chapters: 00:00 Introduction to the episode 00:37 Meet Jack VanLightly: Principal Technologist at Confluent 02:17 Jack's Journey into Distributed Systems 04:29 Discovering the Power of Formal Methods 08:11 Modeling and Simulation in Formal Methods 13:43 Verification and Safety Properties 19:02 Documentation and Communication Challenges 20:43 Formal Methods as a Systems Learning Tool 24:26 Practical Applications and Case Studies 56:38 Future of Formal Verification and Closing Thoughts Jack's Blog: https://jack-vanlightly.com/ Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me. Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription. https://app.codecrafters.io/join?via=geeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!