I am not so good at asking questions here so please let me know if any detail is missing.
I am trying to run a Python script ("o.py") multiple times in a loop from another script ("s.py"). o.py is in a subfolder inside the directory where s.py is located. I want to access a variable that I set in s.py (variable "l") in o.py. But since I am running o.py as a subprocess (I cannot run it any other way as I did not develop this part) I cannot import the variable from s.py to o.py because of some constraints. So as a work around I would like to run the script o.py from s.py with "l" as the input variable. I would like to know how can I accomplish this in both s.py and o.py files. I read some answers but everyone has mentioned the code for the subprocess function, but nothing for receiving the variable in the file being executed through the subprocess (in this case o.py).
Please note that I need to parallelize this process (multiprocessing) and every time, for each process the value of "l" should not change until the subprocess is finished.
Here is a sample of how s.py and o.py look like:
s.py
import multiprocessing as mp
import subprocess
def processToParallelize():
subprocess.call(["python_path" , "o.py"], input = l, text=true)
subprocess.call(["python_path" , "o.py"], input = l, text=true)
subprocess.call(["python_path" , "o.py"], input = l, text=true)
if __name__ == '__main__':
NUMBER_OF_TASKS =2
pool = mp.Pool(NUMBER_OF_TASKS)
pool.map(processToParallelize, range(4))
pool.close()
pool.join()
o.py
r = # this should be equal to "l"
x = someoperation(r)
The other questions that were asked related to this topic did not share the information about how to receive the input argument in the subprocess files. That was an important part which I needed to be resolved.
Since your data is a simple string, use a command argument:
def processToParallelize():
subprocess.call(["python_path" , "o.py"], args=(l,))
o.py
import sys
l = sys.argv[1]
print(l)
It's also common to use environment variables:
import sys
def processToParallelize():
sys.environ["l"] = l
subprocess.call(["python_path" , "o.py"])
o.py
import sys
l = sys.environ["l"]
print(l)