Modern, ultra fast PostgreSQL engineered from scratch? ft: CedarDB

For memberships: join this channel as a member here:https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/joinSummaryIn this conversation, Philipp discusses the innovations behind CedarDB, a database system designed from scratch to optimize performance for modern hardware. He explains the foundational principles of compiling SQL to machine code, the importance of parallel processing, and the challenges of maintaining Postgres compatibility. The discussion also covers the system's approach to handling transactional and analytical workloads, data ingestion processes, query optimization strategies, and future developments including schema evolution and disaggregated storage.Takeaways:- CedarDB is built from the ground up to utilize modern hardware effectively.- The system compiles SQL directly to machine code for performance.- Parallel processing is a key feature, allowing efficient use of multiple cores.- CedarDB aims to be Postgres compatible while innovating on performance.- Transactional workloads are handled efficiently without sacrificing analytical capabilities.- Data ingestion is optimized for both row-oriented and columnar formats.- The system uses optimistic concurrency control to manage write conflicts.- Query optimization leverages statistics to improve join performance.- Future developments include schema evolution and disaggregated storage.- CedarDB is designed to be flexible and adaptable for various workloads.Chapters00:00 Introduction to CDRDB and Background of Philipp05:36 Compiling SQL to Machine Code for Performance11:25 General Purpose vs. Analytical Databases16:51 Transactional Workloads and Hybrid Storage Engine54:29 Understanding B-Tree and Columnar Storage01:02:18 Data Duplication and Memory Efficiency01:08:43 Indexing Strategies and B-Tree Optimization01:15:57 Handling Write Conflicts and Transaction Management01:24:10 Query Optimization and Join Strategies01:33:28 Future Developments in Schema Evolution and StorageImportant Links:CedarDB: https://cedardb.com/The Umbra research project: https://umbra-db.com/SQL Query Compilation: http://www.vldb.org/pvldb/vol4/p539-neumann.pdfOptimistic B-Trees: https://cedardb.com/blog/optimistic_btrees/Our B-Tree storage engine: https://cedardb.com/blog/colibri/For memberships: join this channel as a member here:https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/joinDon't forget to like, share, and subscribe for more insights!=============================================================================Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.https://app.codecrafters.io/join?via=geeknarrator=============================================================================Database internals series: https://youtu.be/yV_Zp0Mi3xsPopular playlists:Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_dModern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsNStay Curios! Keep Learning!