Python multiprocessing Pool on Windows 8.1 spawns only one worker

Question

I currently have this piece of code (feel free to comment on it too :) )

def threaded_convert_to_png(self):
    paths = self.get_pages()
    pool = Pool()
    result = pool.map(convert_to_png, paths)
    self.image_path = result

On an Intel i7 it spawns eight workers when running on Linux; however, when running Windows 8.1 Pro it only spawns one worker. I checked and cpu_count() returns 8 on both Linux and Windows.

• Is there something I am missing here, or doing wrong?
• Is there a way to fix that problem?

P.S. This is in Python 2.7.6

Answer 1

There is one easy way to determine what is happends in your pool - to turn on multiprocessing debug. You can do it like this:

import logging
from multiprocessing import util

util.log_to_stderr(level=logging.DEBUG)

And on script running you will get full info about processes running, spawning and exiting.

But any way, process pool always spawn N processes (where is N - "processes" argument value or cpu_count), but tasks distribution between processes can be uneven - it depends on task run time.