Why does python keep buffering stdout even when flushing and using -u?

Question

$ cat script.py
import sys

for line in sys.stdin:
    sys.stdout.write(line)
    sys.stdout.flush()

$ cat script.py - | python -u script.py

The output is right but it only starts printing once I hit Ctrl-D whereas the following starts printing right away :

$ cat script.py - | cat

which led me to think that the buffering does not come from cat.

I managed to get it working by doing :

for line in iter(sys.stdin.readline, ""):

as explained here : Streaming pipes in Python, but I don't understand why the former solution doesn't work as expected.

Answer 1

Python manpage reveals the answer to your question:

   -u     Force stdin, stdout and stderr to be totally unbuffered.  On systems where it matters, also put stdin, stdout and stderr in binary mode.  Note that
          there  is  internal  buffering  in  xreadlines(),  readlines()  and file-object iterators ("for line in sys.stdin") which is not influenced by this
          option.  To work around this, you will want to use "sys.stdin.readline()" inside a "while 1:" loop.

That is: file-object iterators' internal buffering is to blame (and it doesn't go away with -u).