I'm trying to use Python in a module for an analysis software of vehicle bus systems. For this I have to embed Python in a thread safe manner, since there can be multiple instances of the module witch work independently. I could use a mutex to guard all access to Python and create an unique (python) module for every thread. Obviously this is the easiest way out but comes with the price of not being able to scale across multiple cores. Or I could modify my module to spawn new processes which intern uses Python and connect to them via shared memory. This gives me a performance penalty and costs more time to implement but scales great.
My question: witch one do you think makes more sense? Is there any other way to embed Python thread safe or even in a way that scales over multiple cores.
Kind regards Moritz
edit: I'm using CPython
To follow-up on my question: I went ahead in implemented it using Processes with intern use Python. A good text why the multiprocessing library does not help can be found here: http://pkaudio.blogspot.com/2010/04/whey-multiprocessing-doesnt-always-work.html It was not written by myself but that guy has the same problem as I have. I'm thankful for everybody who tried to help me.