The Backend Engineering Show with Hussein Nasser

Cloudflare identified several limitations in NGINX architecture and decided to write their own reverse proxy. 0:00 Intro 1:53 What NGINX is used for 3:37 NGINX Architecture 7:52 NGINX Limitations 17:12 Cloudflare Pingora Buy me a coffee if you liked this https://www.buymeacoffee.com/hnasr Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Introduction to NGINX (link redirects to udemy with coupon) https://nginx.husseinnasser.com Python on the Backend (link redirects to udemy with coupon) https://python.husseinnasser.com Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Resources Resources: https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/amp/

In this episode of the backend engineering show I discuss the evolution of multi-threading apps, their pros and cons and then I go through 5 threading model and how they interleave with backend connection management between the threads and requests handlings. Enjoy To learn more about networking fundamentals check out my udemy course Fundamentals of Networking for Effective Backends Head to https://network.husseinnasser.com for a discount coupon 0:00 Intro 2:00 Single Threading 6:30 Multi-Threading 14:15 Connection Listener 20:15 How Connections are Established 29:00 Single Listener/Worker thread 33:30 Single Listener, Multiple Worker threads 39:00 Single Listener, Multiple Workers with load balancing 42:10 Multiple Listeners on the same port (SO_REUSEPORT) 45:20 Multiple Single Threaded Backend Buy me a coffee https://www.buymeacoffee.com/hnasr

Memcached is an in memory cache with one major feature be a transient cache. Memcached has a very simple design. It was originally designed to help with database load by storing the query result in memory to avoid further querying the database. By default it has no authentication, a simple text protocols, servers don’t talk to each other. This video discuss the architecture of the cache, design choices and have some critics of the design choices. I go through a demo at the end using docker, telnet and nodes. Enjoy 0:00 Intro 4:40 What is Memcached? 7:45 Memory management 16:00 LRU 25:17 Threading and Connections 30:40 Read Example 34:30 Write Example 36:17 Write and Read collisions 39:40 Locking 40:30 Distributed Cache 43:30 Memcached with Docker/Telnet/NodeJS 45:00 Spin up a Memcached Docker container and telnet 52:17 Memcached and NodeJS 56:15 Four Memached Servers with NodeJS 01:01:00 Summary Resources https://www.cloudflare.com/learning/ddos/memcached-ddos-attack/ https://holmeshe.me/understanding-memcached-source-code-IV/ https://github.com/memcached/memcached/blob/master/doc/protocol.txt https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-threads.html https://holmeshe.me/understanding-memcached-source-code-I/ https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-memory.html https://support-acquia.force.com/s/article/360005256114-Memcached-in-detail https://www.alibabacloud.com/blog/redis-vs-memcached-in-memory-data-storage-systems_592091 https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf https://memcached.org/blog/persistent-memory-2/ https://memcached.org/blog/modern-lru/ Buy me a coffee https://www.buymeacoffee.com/hnasr

In this episode of the backend engineering show I go through the main job of the network interface controller (NIC for short) and how the datacenter is pushing it to the limit by allowing it to do more TCP/IP processing, creating what is being popularized as smartNIC. 0:00 Intro 1:20 What is a NIC? 3:40 NIC job 8:00 When does the OS get involved 12:40 Promiscuous mode 14:00 SmartNIC 18:30 Disadvantages Resources https://developer.nvidia.com/networking/ethernet-adapters https://www.theregister.com/2022/08/11/smartnics_network_market/ https://arxiv.org/abs/1803.09615 Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com

In this episode of the backend engineering show I discuss consistent hashing a very important algorithm in distributed computing specially in database systems such as Apache Cassandra and DynamoDB. 0:00 Intro 2:00 Problem of Distributed Systems 5:00 When to Distribute 7:00 Simple Hashing 9:30 Where Simple Hashing Breaks 11:40 Consistent Hashing 18:00 Adding a Server 21:15 Removing a Server 22:30 Limitations Buy me a coffee https://www.buymeacoffee.com/hnasr

In this episode of the backend engineering show I go through and discuss the Homa Protocol paper which attempts to replace TCP as a protocol in the data centers. I learned a lot from this paper, I have my criticisms of certain aspects, timestamps for topics discussed below. It appears there is a path to replace TCP in the datacenter and professor John tries to explain this path. Referenced materials mentioned in the episode Overview paper https://web.stanford.edu/~ouster/cgi-bin/papers/replaceTcp.pdf Homa 2018 paper (Details) https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf NIC Offloading in Linux https://en.wikipedia.org/wiki/TCP_offload_engine#Support_in_Linux Curl disabling Nigel Algo https://github.com/curl/curl/commit/4732ca5724072f132876f520c8f02c7c5b654d9 0:00 Intro 3:00 The nature of networking data center 5:30 TCP Segments 7:30 There is no “Request” in TCP 12:00 What so unique about Data centers? 14:00 Message Throughput vs Data throughput 18:25 Congestion Control 22:38 Homa’s Congestion Control 25:00 Server Core Load Balancing 28:30 NIC offloading 30:00 Everything Wrong about TCP 37:00 Why not QUIC? 40:00 Limitation of Streaming 44:10 Load Balancing Stream Reading 47:15 Can we treat Segments as Messages? 51:00 Dispatching Messages is Easier 53:00 Connection Orientation 1:00:00 Sender Driven Congestion Control 1:03:00 In Order Packet Delivery 1:07:00 DCTCP 1:08:30 Homa is Message Based 1:11:00 Home is Connection Less 1:12:00 Receiver Driven Congestion Control 1:15:19 Out of Order Packets 1:16:20 Homa API is not Compatible with TCP 1:17:40 Will Homa come to HTTP? 1:18:45 Conclusion

ByteDance, the company behind TikTok is proposing few patches to the linux kernel to make kernel reboots via kexec go from 500ms down to 15 ms saving huge time in updating kernel on thousands of machines. Let us discuss this 0:00 Intro 1:30 Linux Kernel Reboot Options 2:30 how kexec works 4:00 The optimization 5:00 Going through the patch 6:00 Updating Servers at TikTok scale 9:00 Summary https://www.phoronix.com/news/Bytedance-Faster-Kexec-Reboot https://lore.kernel.org/lkml/20220725083904.56552-1-huangjie.albert@bytedance.com/

We discuss the CVE-2022-2274 OpenSSL Vulnerability. The OpenSSL 3.0.4 release introduced a serious bug in the RSA implementation for X86_64 CPUs supporting the AVX512IFMA instructions. This issue makes the RSA implementation with 2048 bit private keys incorrect on such machines and memory corruption will happen during the computation. As a consequence of the memory corruption an attacker may be able to trigger a remote code execution on the machine performing the computation. 0:00 Intro 1:00 CVE-2022-2274 3:00 AVX512IFMA CISC 5:00 How the bug works 7:10 How can it be triggered Resources https://www.openssl.org/news/secadv/20220705.txt https://github.com/openssl/openssl/issues/18625 https://guidovranken.com/2022/06/27/notes-on-openssl-remote-memory-corruption/ https://eprint.iacr.org/2018/335 https://github.com/openssl/openssl/commit/4d8a88c134df634ba610ff8db1eb8478ac5fd345 https://linux.die.net/man/3/bn_internal https://www.microfocus.com/documentation/enterprise-developer/ed60/ES-WIN/GUID-E3960B1E-C42E-4748-A5EB-6E12507C9CD7.html https://www.microcontrollertips.com/risc-vs-cisc-architectures-one-better/ Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com

Postgres 15 introduces a new syntax to treat nulls as not distinct. This will allow developers to create a unique constraint on nullable fields and have inserts with nulls fail instead of having it allowed based on today's standard. Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com https://blog.rustprooflabs.com/2022/07/postgres-15-unique-improvement-with-null https://www.postgresql.org/message-id/flat/84e5ee1b-387e-9a54-c326-9082674bde78%40enterprisedb.com

memchr() is a function that searches in a memory block for a character, this has been recently improved in the latest linux patch and we can learn something from this improvement I think, lets discuss. https://www.phoronix.com/scan.php?page=news_item&px=Linux-Kernel-Faster-memchr Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com