Python: Threading within MultiProcess Function runs in the same core as the multiprocess one or the parent one?

Question

def multiprocess_function():
   run = 0   
   while run == 0:
     for i in the range(100):

        #This will initiate 100 threads
        threading.Thread(target=sum, args=(i,0))

     time.sleep(10)

p1 = multiprocessing.Process(target=multiprocess_function)
p1.start

In the above code snippet, I am starting a new infinite loop process (on a separate core (say #2)). Within this function, I launch 100 threads. Will the threads run on the same core #2 or it will run on the main python core?

Also, how many threads can you run on one core?

Answer 1

All threads run within the process they started from. In this case, the process p1.

With regard to how many threads you can run in a process, you have to keep in mind that in CPython, only one thread at a time can be executing Python bytecode. This is enforced by the Global Interpreter Lock ("GIL"). So for jobs that require a lot of calculations it is generally better to use processes and not threads.

If you look at the documentation for concurrent.futures.ThreadPoolExecutor, by default the number of worker threads that it uses is five times the amount of physical processors. That seams to be a reasonable amount for the kinds of workloads that the ThreadPoolExecutor is meant for.