execve does not inherit any filedescriptor

Question

I'm writing a shell and try on implement process substitution. fork inherit of all filedescriptor, allocated memory etc. I understood that execve should keep also this kind of information, and so keep each opened filedescriptor, whenever O_CLOEXEC flag is not set.

I tried a simple python script :
fd.py :

#!/usr/bin/env python3
import sys, os

if __name__ == "__main__":
    if len(sys.argv) == 1:
        new_fd = open("the_content_file", "w+")
        print("father table : ", os.listdir("/dev/fd"))
        if os.fork() == 0:
            os.execve("/PATH/OF/SCRIPT/fd.py", ["fd", "content"], os.environ)
    else:
        print("child table : ", os.listdir("/dev/fd"))
    pass

and as output, I get :

father table :  ['0', '1', '2', '3', '4']
child table :  ['0', '1', '2', '3']

After the fork, I keep the same fd table, but whenever I'm using execve on an executable, I lost all and get the fd opened by default. Why the opened fd is disappearing ? Thanks

Answer 1

python3 (since version 3.4, and unlike python2) is opening files with the O_CLOEXEC flag by default.

I'm no python programmer, but an easy way to turn O_CLOEXEC back off on a file could be by adding just after the new_fd = .. line:

        os.set_inheritable(new_fd.fileno(), True)

(tested on python 3.6.6, see the docs here)

or, on older versions like 3.5.3:

        tmp_fd = os.dup(new_fd.fileno())
        os.dup2(tmp_fd, new_fd.fileno())
        os.close(tmp_fd)

(os.dup2 used to make the destination fd inheritable by default)

Notice that despite the name you have given it, your new_fd is not a file descriptor, but a python stream. The extra file you see in both the parent and the child is an open handle to the /dev/fd directory.