The Backend Engineering Show with Hussein Nasser cover image

How Discord Stores Trillions of Messages - A deep dive

The Backend Engineering Show with Hussein Nasser

00:00

Using a Load Balancing to Consolidate Requests

It's a worker subscriber thing where the first Person who made the request will create this worker and then send a message while all of these queries at the same time Concurrently we're gonna look up this worker. As long as the worker is executing you're gonna hook to it I'm but what happened if the workers get their response and rights is the worker dead? Do we cash the worker result for subsequent queries or do we create new workers every time? The second part of the magic here And this is truly magic. We implemented consistent Hash based routing to our data service to enable more effective coalescing. It was able to significantly reduce traffic spikes. Previously this might have a hot partition and

Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner