I am currently working in Python. I need to work with multiprocessing.Pool and set the current working directory as the file dir an then adding the parent directory to sys.path. The problem is that, when multiprocessing.Pool starts, it runs again the line sys.path.insert(0,'..')
which I want to avoid. Is there any way to make multiprocessing.Pool not running the Globals or anything which is outside the if __name__ == "__main__":
snippet of code?
import os
import sys
import multiprocessing
os.chdir(sys.path[0])
sys.path.insert(0,'..')
print(os.getcwd())
def Worker(j):
pass
if __name__ == "__main__":
with multiprocessing.Pool(1) as p:
p.map(Worker, range(10))
p.close()
p.join()
So the way multiprocessing works is that when it hits the p.map(Worker, range(10))
line, it would start a new child process, and run the whole script again from line 1 (new child process would import the calling module). The lines under the if
would not run again in child process. To avoid the sys.path.insert(0,'..')
to be repeated in the child process, you need to put that line under the if
.
See this for more details.
import os
import sys
import multiprocessing
def Worker(j):
pass
if __name__ == "__main__":
os.chdir(sys.path[0])
sys.path.insert(0,'..')
print(os.getcwd())
with multiprocessing.Pool(1) as p:
p.map(Worker, range(10))
p.close()
p.join()