python subprocess.Popen throwing error while after a Thread is activated

Question

I looked around, but I didn't see any information on this particular problem so I thought I'd ask it here:

Whenever I try to run a subprocess.Popen call after starting up a random thread, I get an OSError. My assumption is that there is some aspect to subprocess.py that I don't understand in relation to multithreading.

I am running python 2.7.3 on a QNX system.

Code test:

import subprocess
import time
from threading import Thread

def Test():
  while(1):
     print "Testing."
     time.sleep(1)

if __name__=="__main__":
   subprocess.Popen('ls') # runs just fine

   Thread(target=Test).start()

   subprocess.Popen('ls') # throws OSError

Traceback

Traceback (most recent call last):
   File "test.py", line 16 in <module>
      subprocess.Popen('ls')
   File "usr/local/lib/python2.7/subprocess.py", line 679, in __init__
      errread, errwrite)
   File "usr/local/lib/python2.7/subprocess.py", line 1143, in _execute_child
      self.pid = os.fork()
OSError: [Errno 89] Function not implemented

Answer 1

QNX is not designed to support fork() within a process after the process has already spawned new threads. From the QNX documentation...

Suppose you have a process and you haven't created any threads yet (i.e., you're running with one thread, the one that called main()). When you call fork(), another process is created, also with one thread. This is the simple case.

Now suppose that in your process, you've called pthread_create() to create another thread. When you call fork(), it will now return ENOSYS (meaning that the function is not supported)! Why?

Well, believe it or not, this is POSIX compatible — POSIX says that fork() can return ENOSYS. What actually happens is this: the Neutrino C library isn't currently built to handle the forking of a process with threads. When you call pthread_create(), the function sets a flag, effectively saying, “Don't let this process fork(), because I'm not prepared to handle it.” Then, in the library fork() function, this flag is checked, and, if set, causes fork() to return ENOSYS.

(source, read the Process creation and threads section.)

I'm assuming Thread(target=Test).start() calls pthread_create() and accounts for the error you are seeing.

It is also worth noting the definition of ENOSYS also specifies the function is not implemented. See the following definition...

ENOSYS Function not implemented (POSIX.1-2001).

(source)

Thus, this is an OS level constraint. Not an issue with Python!