Difference between subprocess.Popen preexec_fn and start_new_session in python

Question

What is the difference between these two options to start a new process with subprocess.Popen for python3.2+ under Linux:

proc = subprocess.Popen(args, ..., preexec_fn=os.setsid)   # 1
proc = subprocess.Popen(args, ..., start_new_session=True) # 2

I need this as I need to set process group ID to have a possibility to kill at once this process and all of its children. This is then used in the case if the process run time exceeds certain threshold:

try:
    out, err = proc.communicate(timeout=time_max)
except subprocess.TimeoutExpired:
    os.killpg(os.getpgid(proc.pid), signal.SIGTERM) 

I tested my code with the both options (#1 & #2) and they both seem to work ok for me.

But I wonder what is the best option here - the one with preexec_fn or the one with start_new_session?

Answer 1

According to the official Python Docs,

The preexec_fn parameter is not safe to use in the presence of threads in your application. The child process could deadlock before exec is called. If you must use it, keep it trivial! Minimize the number of libraries you call into.

If you need to modify the environment for the child use the env parameter rather than doing it in a preexec_fn. The start_new_session parameter can take the place of a previously common use of preexec_fn to call os.setsid() in the child.

So I guess the answer to your question is that start_new_session was introduced to replace the common operation of using preexec_fn to set the session id through os.setsid(), which is not thread safe.