Search code examples
pythonpycharmpython-multiprocessing

Exception thrown on pool.close() while debugging, but not while running


I don't think I encountered this problem working on this in Python 2.7, but while debugging in 3.7, Python throws an exception when pool.close() is called. This is the relevant part of the function:

pool = multiprocessing.Pool(6)
iterator = pool.imap_unordered(worker_func, worker_input)

while True:
    try:
        t0, t1 = next(iterator)
    except multiprocessing.TimeoutError:
        continue
    except StopIteration:
        break
    else:
        dbinserts1(t0)
        dbinserts2(t1)            

pool.close()
pool.join()

The only change made by 2to3 was rewriting iterator.next() as next(iterator). The function only fails while debugging (in PyCharm), otherwise it runs successfully. This is (probably) the most relevant part of the stack trace:

Error in atexit._run_exitfuncs: Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/util.py", line 322, in _exit_function p.join() File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 138, in join assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process


Solution

  • Which PyCharm version do you use? This seems to be fixed in 2019.1.2 by https://youtrack.jetbrains.com/issue/PY-34436