The Backend Engineering Show with Hussein Nasser

This new PostgreSQL 17 feature is game changer. They know can combine IOs when performing sequential scan. Grab my database coursehttps://courses.husseinnasser.com

No technical video today, just talking about the idea of discipline and consistency.

Fundamentals of Operating Systems Course This video is an overview of how the operating system kernel does socket management and the different data structures it utilizes to achieve that.timestamps0:00 Intro1:38 Socket vs Connections7:50 SYN and Accept Queue18:56 Socket Sharding23:14 Receive and Send buffers27:00 Summary

Discover the fascinating world of long polling and its role in backend efficiency. Explore how it improves upon traditional polling, reducing latency and resource consumption. Delve into its benefits for scalable database management, emphasizing efficient data sorting and predictable system behavior. Learn about request resumption and how to harness the full potential of long polling to enhance your applications. This insightful discussion uncovers the genius behind a seemingly simple yet powerful design pattern.

Explore the six stages of becoming a better software engineer. Discover how deep understanding transforms your relationship with technology. Learn the art of articulating technical concepts without jargon. Delve into the emotional journey of facing inherent limitations in technology. Gain insights into software efficiency through the lens of operating systems. Ultimately, appreciate the beauty in both the flaws and strengths of technology as you navigate your path in engineering.

A new Linux kernel patch promises to enhance network performance by implementing zero-copy techniques. Instead of the traditional method, this update turns data reading into a notification process, allowing userspace to directly access the data's location. The conversation dives into the intricacies of memory management and how this innovation could revolutionize data transmission efficiency in operating systems. Curious about its impact on performance? They break down the benefits and implications of this groundbreaking approach.

Discover how Cloudflare revolutionized its CDN caching with a global cache purge that executes in under 150 ms. The shift from a centralized lazy purge to a decentralized active system showcases remarkable engineering feats. Explore the intricacies of RockDB, peer-to-peer data structures, and how these innovations address previous limitations. Delve into the performance implications of LSM versus BTree and unpack the benefits of immediate purge requests. This deep dive reveals the complexities and clever strategies behind cutting-edge backend engineering.

Fundamentals of Database Engineering udemy course https://databases.win MySQL has been having bumpy journey since 2018 with the release of the version 8.0. Critical crashes that made to the final product, significant performance regressions, and tons of stability and bugs issues. In this video I explore what happened to MySql, are these issues getting fixed? And what is the current state of MySQL at the end of 2024. 0:00 Intro 2:00 MySQL 8.0 vs 5.7 Performance 11:00 Critical Crash in 8.0.38, 8.4.1 and 9.0.0 15:40 Is 8.4 better than 8.0.36? 16:30 More Features = More Bugs 22:30 Summary and my thoughts resources https://x.com/MarkCallaghanDB/status/1786428909376164263 https://www.percona.com/blog/do-not-upgrade-to-any-version-of-mysql-after-8-0-37/ http://smalldatum.blogspot.com/2024/09/mysql-innodb-vs-sysbench-on-large-server.html https://www.percona.com/blog/mysql-8-0-vs-5-7-are-the-newer-versions-more-problematic/

Fundamentals of Operating Systems Course https://oscourse.win In this video I use strace a performance tool that measures how many system calls does a process makes. We compare a simple task of reading from a file, and we run the program in different runtimes, namely nodejs, buns , python and native C. We discuss the cost of kernel mode switches, system calls and pe 0:00 Intro 5:00 Code Explanation 6:30 Python 9:30 NodeJS 12:30 BunJS 13:12 C 16:00 Summary

Fundamentals of Operating Systems Course https://os.husseinnasser.com When do you use threads? I would say in scenarios where the task is either 1) IO blocking task 2) CPU heavy 3) Large volume of small tasks In any of the cases above, it is favorable to offload the task to a thread. 1) IO blocking task When you read from or write to disk, depending on how you do it and the kernel interface you used, the write might be blocking. This means the process that executes the IO will not be allowed to execute any more code until the write/read completes. That is why you see most logging operations are done on a secondary thread (like libuv that Node uses) this way the thread is blocked but the main process/thread can resume its work. If you can do file reads/writes asynchronously with say io_uring then you technically don't need threading. Now notice how I said file IO because it is different than socket IO which is always done asynchronously with epoll/select etc. 2) CPU heavy The second use case is when the task requires lots of CPU time, which then starves/blocks the rest of the process from doing its normal job. So offloading that task to a thread so that it runs on a different core can allow the main process to continue running on its the original core. 3) Large volume of small tasks The third use case is when you have large amount of small tasks and single process can't deliver as much throughput. An example would be accepting connections, a single process can only accept connections so fast, to increase the throughput in case where you have massive amount of clients connecting, you would spin multiple threads to accept those connections and of course read and process requests. Perhaps you would also enable port reuse so that you avoid accept mutex locking. Keep in mind threads come with challenges and problems so when it is not required. 0:00 Intro 1:40 What are threads? 7:10 IO blocking Tasks 17:30 CPU Intensive Tasks 22:00 Large volume of small tasks