rotate (reopen) file in python3, when signal is received

Question

I have simple script, that logs to a logfile. This is the core of my script:

with open('/var/log/mail/mail.log', 'a', buffering=1) as f:
    for line in sys.stdin:
        f.write(line)

The file being written /var/log/mail/mail.log has to be rotated regularly by logrotate. At the moment, when logrotate rotates the file, my script does not realize it, and continues writing to the old (now renamed) file.

logrotate has the possibility to execute command after the file has been rotated. Normally, when rsyslog is logging, the command would be:

postrotate
    invoke-rc.d rsyslog rotate > /dev/null
endscript

But in my case, I need to send some signal to my script, and handle the signal in my script.

Also, I don't know in advance the PID my script is running as.

How can I best implement this ?

Answer 1

As a solution for this you can watch if the inode of the open log file is the same as the path. If not reopen the file. This only works on Unix.

import os, sys, stat

logfile = '/var/log/mail/mail.log'

while True:
    with open(logfile, 'a', buffering=1) as f:
        f_ino = os.stat(logfile)[stat.ST_INO]
        f_dev = os.stat(logfile)[stat.ST_DEV]
        for line in sys.stdin:
            f.write(line)
            try:
                if os.stat(logfile)[stat.ST_INO] != f_ino or 
                   os.stat(logfile)[stat.ST_DEV] != f_dev:
                     break
            except OSError: # Was IOError with python < 3.4
                pass
            

Closing the file is not required as it’s handled by the with context manager.

The try..except OSError block is used to catch any error by the system function os.stat. It can be that during the change of the file the function returns an OSError (For example a FileNotFoundError). In this case it will pass the exception and retry the check if the inode is changed. If we omit this try..except block you might end up with a program that terminates.