Search code examples
pythonselectsubprocesspipeblocking

Why python's subprocess.PIPE is not blocking?

Consider the below code:

import os, select, subprocess
p1 = subprocess.Popen(["pwd"], stdout = subprocess.PIPE)
r, _, _ = select.select([p1.stdout.fileno()], [], [])
os.read(r[0], 1024)
r, _, _ = select.select([p1.stdout.fileno()], [], [])
os.read(r[0], 1024)

The select function properly reports that the p1.stdout pipe has data in it but the second time while it reports the same, this pipe has no data. Everyone's problem is that the subprocess pipes are blocking. My problem is that this pipe is not blocking. I want the select function to block until the pipe has data in it.

Here is the output:

>>> import os, select, subprocess
>>> p1 = subprocess.Popen(["pwd"], stdout = subprocess.PIPE)
>>> r, _, _ = select.select([p1.stdout.fileno()], [], [])
>>> os.read(r[0], 1024)
b'/tmp\n'
>>> r, _, _ = select.select([p1.stdout.fileno()], [], [])
>>> os.read(r[0], 1024)
b''
>>>
Solution

  • The subprocess has exited, so the write end of the pipe is closed. When you try to read from the pipe again, you're at EOF, so an empty string is returned.

    The documentation of Methods of File Objects explains this:

    If the end of the file has been reached, f.read() will return an empty string

    select() only blocks if trying to read would have blocked. You don't block when you're at EOF, it returns the empty string immediately.