There behaviour of the following minimal code seems incorrect. Why does the second select.select fail to find the remaining line. Is there a buffering somewhere and how do I expose the correct buffer?
import select
import os
read, write = os.pipe()
writeable = os.fdopen(write, "w")
readable = os.fdopen(read, "r")
writeable.write("first\n")
writeable.write("second\n")
writeable.flush()
if select.select([readable], [], [], 10)[0][0] == readable:
print(readable.readline())
print(str(select.select([readable], [], [], 1)))
print(readable.readline())
--- Results:
first
([], [], [])
second
The issue is due to buffering. I couldn't find any concrete documentation but it looks like the full input is being pulled into the buffer when you first call readline.
You can specify no buffering as the third variable in fdopen, but it isn't allowed for text, throwing
ValueError: can't have unbuffered text I/O
If you make the input a byte stream which does allow for unbuffered I/O, you can see the difference (Changes marked with comments):
import select
import os
read, write = os.pipe()
# Use a byte stream and add 0 to disable buffering
writeable = os.fdopen(write, "wb", 0)
readable = os.fdopen(read, "rb", 0)
# Write in bytes
writeable.write(b"first\n")
writeable.write(b"second\n")
writeable.flush()
if select.select([readable], [], [], 10)[0][0] == readable:
print(readable.readline())
print(str(select.select([readable], [], [], 1)))
print(readable.readline())
# Do another check on select.
print(str(select.select([readable], [], [], 1)))
Doing this gives us an output of:
>>>b'first\n'
>>>([<_io.FileIO name=4 mode='rb' closefd=True>], [], [])
>>>b'second\n'
>>>([], [], [])
which I'd guess is the behaviour you were expecting, and if you then remove the disable buffering 0 variable from the fdopen calls,
writeable = os.fdopen(write, "wb")
readable = os.fdopen(read, "rb")
you get back:
>>>b'first\n'
>>>([], [], [])
>>>b'second\n'
>>>([], [], [])
As in your original example.