Search code examples
python-2.7multiprocessingpython-multiprocessingpoolsigterm

python SIGTERM not work when I use multiprocessing Pool


Correct code as expected: from multiprocessing import Pool import signal import time import os

def consumer(i):
    while True:
        # print os.getpid()
        pass
def handler(signum, frame):
    print 'Here you go'

signal.signal(signal.SIGTERM, handler)
p = Pool(5)
p.map_async(consumer, [1 for i in range(5)])

while True:
    pass

p.terminate()
# p.close()
p.join()

==================================================

I have found the problem, when I use map function, the main func is blocked, and signal handler will be called only when map function is funished. So, use "map_async" function is much more better to fix this problem.

Here is what I found:

A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.

==================================================

I wrote a program like the following, and I expect to exit/(like the program print string) in the program when I type "kill pid" in the terminal, but it not work. Is there any other strategy that block the SIGTERM get in the main func.

from multiprocessing import Pool
import signal
import time
import os

def consumer(i):
    while True:
        # print os.getpid()
        pass
def handler(signum, frame):
    print 'Here you go'

signal.signal(signal.SIGTERM, handler)
p = Pool(5)
p.map(consumer, [1 for i in range(5)])

p.terminate()
# p.close()
p.join()

Solution

  • You need to use the map_async method as the map one blocks until results are not ready. Therefore, in your example the call to terminate is never reached.