The GeekNarrator

In this video I talk to Jayaprabhakar Kadarkarai aka JP who is the founder of FizzBee. FizzBee is a design specification language and model checker to help developers verify their design before writing even a single line of implementation code. We have discussed where it is applicable, what are the benefits, how does it work and many other interesting challenges with examples. Chapters: 00:00 Introduction 01:13 Challenges in Designing Distributed Systems 03:13 Understanding Design Specification Languages 04:00 The Value of Structured Design Documents 09:00 When to Use Design Specification Languages 21:27 Modeling a Travel Booking System 22:51 Ensuring Atomicity in Distributed Systems 26:09 Handling Failures and Consistency 34:45 Refinement in System Design 35:38 Balancing Abstraction and Implementation 37:53 Common Pitfalls in Modeling and Implementation 40:02 Challenges in System Design and Implementation 40:12 Two-Way Feedback in System Design 41:01 Performance Considerations in Implementation 41:36 Importance of Solid Design Blueprints 41:56 Model-Based Testing and Continuous Integration 43:27 Updating Design Documentation 44:38 Simulation Testing vs. Model Checking 45:32 Design Issues and Formal Verification 49:51 Applying Formal Verification to Existing Systems 55:35 Common Design Problems and Solutions 01:07:57 Future Enhancements in Design Specification Tools 01:12:50 Getting Started with FizzBee FizzBee : https://fizzbee.io/ Get in touch with JP: https://www.linkedin.com/in/jayaprabhakar Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription. https://app.codecrafters.io/join?via=geeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #systemdesign #formalmethods

Kishore Gopalakrishna, Co-founder and CEO of StarTree and co-author of Apache Pinot, shares his wealth of knowledge in real-time analytics and distributed systems. He reveals the challenges and innovations involved in building systems like Apache Pinot and discusses the pivotal role of community in open-source success. Kishore also delves into effective cost optimizations and the transition from local to cloud storage, emphasizing how real-time analytics can transform data-driven decision-making in businesses.

In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Ryan and Richie, the founders of WarpStream. They discuss the architecture, benefits, and core functionalities of WarpStream, a drop-in replacement for Apache Kafka. The conversation covers their experience with Kafka, the design decisions behind WarpStream, and the operational challenges it addresses. They also delve into the seamless migration process, the scalability, and cost benefits, the integration with the Kafka ecosystem, and potential future features. This episode is a must-watch for developers and tech enthusiasts interested in modern, distributed data streaming solutions. Chapters: 00:00 Introduction 02:27 Introducing Warpstream: A Kafka Replacement 11:07 Deep Dive into Warpstream's Architecture 35:42 Exploring Kafka's Ordering Guarantees 36:52 Handling Buffering and Compaction 38:44 Efficient Data Reading and File Caching 44:06 WarpStream's Flexibility and Cost Efficiency 01:06:59 Future Features Links: WarpStream : https://www.warpstream.com/ Blog: https://www.warpstream.com/blog X: Ryan: https://x.com/ryanworl Richard Artoul: https://x.com/richardartoul Kaivalya Apte: https://x.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #kafka #s3 #streaming

Exploring XTDB with Jeremy Taylor & Malcolm Sparks: An In-Depth Dive into Immutability and Database Internals In this episode of the Geek Narrator Podcast, host Kaivalya is joined by Jeremy Taylor and Malcolm Sparks from Juxt to explore XTDB, an immutable database designed to handle complex historical and financial data with precision. They delve into the architecture, internal mechanics, and use cases while discussing the importance of immutability. This episode covers everything you need to know about XTDB and its capabilities. Whether you're a developer interested in databases or someone curious about data management and history tracking, this discussion offers invaluable insights. Chapters: 00:00 Introduction 02:51 Challenges with General Purpose Databases 11:50 XTDB: A New Approach to Databases 31:56 Understanding Kafka and XTDB Integration 36:06 Querying and Indexing in XTDB 40:31 Temporal Data Management and Use Cases 54:52 Deployment and User Experience XTDB: https://xtdb.com/ XTDB Github: https://github.com/xtdb/xtdb Juxt: https://www.juxt.pro/ Juxt Github: https://github.com/juxt If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #sql #kafka #datastorage #immutable

Will Wilson, Engineer and co-founder of Antithesis, dives deep into the world of deterministic simulation testing for distributed systems. He breaks down the limitations of traditional methods, showcasing how his company's approach improves software reliability. Key discussions include optimizing bug detection strategies, the significance of simulated workloads, and the challenges posed by third-party APIs. Real-world examples like chat applications illustrate how effective testing can reveal hidden issues, making this an essential listen for tech enthusiasts and developers.

Glauber Costa, founder and CEO of TursoDB, brings his background in Linux kernel development to the forefront as he discusses enhancing SQLite for production use. He shares insights on native replication, schema management, and vector search capabilities. Glauber navigates the challenges of transitioning from unikernel systems and highlights the importance of multi-tenancy. Listeners will also learn about TursoDB's user-friendly design and its future roadmap, including features aimed at improving database simplicity and efficiency.

Deep Dive into Serverless Databases with Neon: Featuring Heikki Linnakangas In this episode of the Geek Narrator podcast, host Kaivalya Apte is joined by Heikki Linnakangas, co-founder of Neon, to explore the innovative world of serverless databases. They discuss Neon's unique approach to separating compute and storage, the benefits of serverless architecture for modern applications, and dive into various compelling use cases. They also cover Neon's architectural features like branching, auto-scaling, and auto-suspend, making it a powerful tool for both developers and enterprises. Whether you're curious about multi-tenancy, fault tolerance, or developer productivity, this episode offers insightful knowledge about leveraging Neon's capabilities for your next project. 00:00 Introduction 00:53 The Birth of Neon: Why It Was Created 02:16 Understanding Serverless Databases 07:06 Neon's Architecture: Separation of Compute and Storage 09:59 Exploring Branching in Neon 18:21 Auto Scaling and Handling Spikes in Traffic 20:17 The Challenge of Multiple Writers in Distributed Systems 22:51 Auto Suspend: Cost-Effective Database Management 26:02 Optimizing Cold Start Times 27:14 Balancing Cost and Performance 28:52 Replication and Durability 30:32 Understanding the Storage Layer 34:02 Custom LSM Tree Implementation 36:21 Fault Tolerance and Failover 07:00 Developer Productivity and Use Cases 42:56 Migration and Tooling 48:35 Future Roadmap and User Experience 50:28 Conclusion and Final Thoughts Neon website: https://neon.tech/ Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #PostgreSQL #SQL #RDBMS #NEON

Felix GV, Principal Staff Engineer at Linkedin, discusses the evolution from Voldemort to VeniceDB, challenges with derived data, planet-scale applications, writing and merging data, architectural components, fault tolerance, recommender systems, versioning, integration with Kafka, and future improvements.

In this video I speak with Philippe Noël, about ParadeDB, which is an Elasticsearch alternative built on Postgres, modernizing the features of Elasticsearch's product suite, starting with real-time search and analytics. I hope you will enjoy and learn about the product. Chapters: 00:00 Introduction 01:12 Challenges with Elasticsearch and the Need for ParadeDB 02:29 Why Postgres? 06:30 Technical Details of ParadeDB's Search Functionality 18:25 Analytics Capabilities of ParadeDB 24:00 Understanding ParadeDB Queries and Transactions 24:22 Application Logic and Data Workflows 25:14 Using PG Cron for Data Migration 30:05 Scaling Reads and Writes in Postgres 31:53 High Availability and Distributed Systems 34:31 Isolation of Workloads 39:38 Database Upgrades and Migrations 41:21 Using ParadeDB Extensions and Distributions 43:02 Observability and Monitoring 44:42 Upcoming Features and Roadmap 46:34 Final Thoughts Important links: Links: GitHub: https://github.com/paradedb/paradedb Website: https://paradedb.com Docs: https://docs.paradedb.com/ Blog: https://blog.paradedb.com Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #postgresql #datafusion #parquet #sql #OLAP #apachearrow #database #systemdesign #elasticsearch

Andrew Lamb, Staff Software Engineer at InfluxDB and chair of the Apache Data Fusion project, shares his expertise on modern OLAP database design. He explains the power of the FDAP stack, highlighting how Apache Parquet and Arrow enhance data storage and retrieval efficiency. The conversation delves into the challenges of data immutability and management, while also discussing Flight's role in simplifying data transfer. Looking ahead, Andrew envisions evolving trends in database technologies, paving the way for innovative solutions in analytics.