So here's the deal: I need a Monte-Carlo simulation to ensure a sensor is as accurate as specified. Problem, it's painfully slow. I won't get into details but I need to do a lot of maths behind the scenes. On top of that, I have to test 1M+ times to get accurate results, and we end up with easily 2h+ of calculation time alone, which obviously isn't really nice.
So, I've been looking around for some sort of parallelism, and multi-processing felt like the perfect solution.
Thus, I tried to pull some code from the official documentation, as follows...
from multiprocessing import Process, Value, Array
def f(n, a):
n.value = 3.1415927
for i in range(len(a)):
a[i] = -a[i]
if __name__ == '__main__':
num = Value('d', 0.0)
arr = Array('i', range(10))
p = Process(target=f, args=(num, arr))
p.start()
p.join()
print(num.value)
print(arr[:])
...but of course,
_pickle.PicklingError: Can't pickle <function f at 0x0000026124EDA3B0>: attribute lookup f on __main__ failed
EOFError: Ran out of input
Visibly, somehow, p.start() causes this. Which, surprised me at first, especially from a code snippet coming directly from the docs.
Anyways, I kept looking and trying out some snippets I could find here and there, but no matter what, I'd keep getting the same two errors.
And I just don't get why? What am I missing out?
Here's the entire error log, from trying to run the snippet above.
Traceback (most recent call last):
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 1, in <module>
File "C:\Program Files\JetBrains\PyCharm 2022.2.3\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 198, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2022.2.3\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "D:\DEV\SYMETRIE\v4\testMP1.py", line 13, in <module>
p.start()
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x0000026124EDA3B0>: attribute lookup f on __main__ failed
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\nicol\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
As it turns out, this is a problem with PyCharm (my IDE of choice) and has been a known bug for now 3 years. Using another IDE fixes this issue. I guess it's time to change my workflow a little bit.
And I found a fix! Head over to your configurations -> Edit -> turn off "Run With Python Console". It's fixed!
For those who can't find it, it's right here
