What is the effect of pydevd.py's -- pydev's debugger used by Eclipsed and Pycahrm -- effect on python's multiprocessing & threading?
When I start my python server -- which uses both multiprocessing & threading -- without debugging, it crashes on startup with no helpful traceback. However, when I run with Pycharm's debugger, the process starts up and functions properly.
So what would be causing this difference in performance? Could it be serialization of Process() or imposing of some orders on Thread()?
Well, it may be that you have some racing condition and just by having the debugger running things run in a different order (it does some monkey-patching to auto-attach to subprocesses and it does many things to reset its state in the new process).
Note that using threading and multiprocessing at the same time can be very tricky if you're starting your sub-processes with a fork on Linux (to resume, deadlocks are expected on corner cases if you do that) -- if you're on Python 2 there's no workaround, but if you're on Python 3 you should be able to set the start method to spawn: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_start_method.
I realize there are use cases where the fork method may be preferred vs spawn (to avoid copying too much memory on some use-cases). If that's your use case you should not use threads to continue using fork or you should be very careful about that interaction and reset anything that a thread could touch in the new process and remove references to all the threads and restart them in the new process (the debugger is multi-threaded and does a lot of things to try to work around that, but I'm still aware that just by having threads on some corner cases it could crash -- seldomly -- just because you're forking with those threads running depending on system level locks).