Multithreading vs Multiprocessing. When to use what?

Multithreading and multiprocessing can be confusing because they aim to achieve the same - run your code faster. There is actually a huge difference between these two and in a way their use is quite opposite so if you mess up one with the other, you can make your code much slower.

I am not going to explain internal detail about CPU architecture since there is no space for it. I will simply focus on the question when to use what (and why) so that you will get a clear idea. It will be demonstrated in a simple example.

So, here is the deal.

Use threading if you have a blocking call in your code. Use multiprocessing if you have complex CPU calculations.

If you do not have blocking calls, don't use threading. If you do not have complex calculations, don't use multiprocessing.

Now, what is a blocking call? By blocking call I mean all kinds of IO stuff (waiting for user input, waiting for a server to respond, waiting for a database response, etc.). It is blocking because the program is blocked. It is stopped at that moment and is waiting, the processor is not occupied, it is resting.

Complex calculations on the other hand is a something where the processor is constantly occupied, having no time to rest, it doesn't wait for anything, it is using all its power to give you the result. Examples are mathematical calculations, text parsing, rendering a graphic, processing audio, etc. The processor is 100% occupied and busy.

So once again.

Blocking call (waiting for something) -> use multithreading.
Complex calculation (processing something) -> use multiprocessing.

Now, let's look at some code.

Here I have two functions. One is blocking, the other is not.

def blocking_function():
    name = input("what is your name? ")  # blocking IO call

def calculating_function():
     [x**10 for x in range(10_000_000)]  # complex calculations

Now imagine I want to run them in sequence. How much time will it take to finish both of them? Simply add the time each of these takes.

If the blocking function will wait 10 seconds for user input and then the calculation will take the other 10 seconds, the total time will be 20 seconds. But if the user input will be given within 1 second, it will take only 11 seconds to finish both.

So let's add some timing there and run it

def blocking_function():
    start = time.time()
    name = input("what is your name? ")  # blocking IO call
    print(f"blocking_function took {time.time() - start} seconds")

def calculating_function():
    start = time.time()
    [x**10 for x in range(10_000_000)]
    print(f"calculating_function took {time.time() - start} seconds")

if __name__ == "__main__":
    start = time.time()
    blocking_function()
    calculating_function()
    print(f"it all together took {time.time() - start} seconds")

I have this output 

blocking_function took 2.0570812225341797 seconds
calculating_function took 11.591603994369507 seconds
it all together took 13.652696371078491 seconds

It took me 2.05 seconds to input my name and the calculation on my laptop took 11.5 seconds. So the total time is the sum of these two.

Now let's see how things change when I run these functions using multithreading. Why multithreading? Because I have a blocking call in the code, right? It will be waiting for input. 

The two functions stay the same, I just need to change the run. Don't bother about the syntax details now, this is just one way to write it.

from threading import Thread

if __name__ == "__main__":
    t1 = Thread(target=blocking_function)  # initialize the threads
    t2 = Thread(target=calculating_function)
    start = time.time()
    t1.start()  # start the threads
    t2.start()
    t1.join()  # wait for them to finish
    t2.join()
    print(f"it all together took {time.time() - start} seconds")

My output is now this.

blocking_function took 4.138643503189087 seconds
calculating_function took 10.763818502426147 seconds
it all together took 10.769804000854492 seconds

Now, it took me about 4.1 seconds to type my name and the calculation took about 10.7 seconds. The total time is now basically equivalent to the time that the calculating function needed to process the task. It did not wait for me to first put the input. It was running while I was typing. And that is the whole point.

Let's contrast it with what happens if I use multithreading for code where there is no blocking call, only complex calculations.

if __name__ == "__main__":
    t1 = Thread(target=calculating_function)  # initialize the threads, both will do complex calculations
    t2 = Thread(target=calculating_function)
    start = time.time()
    t1.start()  # start the threads
    t2.start()
    t1.join()  # wait for them to finish
    t2.join()
    print(f"it all together took {time.time() - start} seconds")

This code is the exact same as the previous, except that BOTH threads perform the complex calculations. When I run it, I got this result.

calculating_function took 23.52871298789978 seconds
calculating_function took 24.031445741653442 seconds
it all together took 24.051433086395264 seconds

As you can see it took double the amount of time. Why is that much slower? Because both the threads need to perform the heavy task, there is no waiting for anything at all, and because I defined it as threaded code, the operating system switches between these two threads, thus adding up more time and making it totally useless.

So here you have it.

Do not use threads for complex calculations. Not only you will not benefit from it. You will actually harm yourself. Python's threads are great only when you need to wait for something. They are perfect at doing nothing.