AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Synchronizing Processes With Forks
With multi threading, you could probably do 60 to 100 simultaneous network requests. But with multiprocessing instead where you have a like a process pool, and you give it a whole bunch of stuff to work on, each process is only going to work on one request at a time. So if you actually really want to saturate all your cores, now you need a whole bunch more processes. And that then has the problem of a lot of memory overhead because even if you're using copy on right semantics with forking, the problem is that Python goes and touches all the ref counts on everything.