Search code examples
pythonpython-3.xstreamgeneratorpipeline

Reading infinite stream - tail


Problem:

Program to read the lines from infinite stream starting from its end of file.

#Solution:

import time
def tail(theFile):
    theFile.seek(0,2)   # Go to the end of the file
    while True:
        line = theFile.readline()
        if not line:
            time.sleep(10)    # Sleep briefly for 10sec
            continue
        yield line

if __name__ == '__main__':
    fd = open('./file', 'r+')
    for line in tail(fd):
        print(line)

readline() is a non-blocking read, with if check for every line.

Question:

It does not make sense for my program running to wait infinitely, after the process writing to file has close()

1) What would be the EAFP approach for this code, to avoid if?

2) Can generator function return back on file close?


Solution

  • @ChristianDean in his comment answered your first question quite nicely, so I'll answer your second.

    I do believe it is possible - you can use theFile's closed attribute and raise a StopIteration exception if the file was closed. Like this:

    def tail(theFile):
        theFile.seek(0, 2) 
        while True:
            if theFile.closed:
                raise StopIteration
    
            line = theFile.readline()
            ...
            yield line
    

    Your loop shall cease when the file is closed and the exception is raised.


    A more succinct method (thanks, Christian Dean) that does not involve an explicit exception is to test the file pointer in the loop header.

    def tail(theFile):
        theFile.seek(0, 2) 
        while not theFile.closed:
            line = theFile.readline()
            ...
            yield line