Monitoring if a file stopped writing in python

Question

I have a program that keeps writing to a file every second. File writing is happening in a thread parallel to the UI. Due to some hardware issue it stops writing sometimes of a day. I wanted to check if the file stopped writing, restart the program if it is not getting updated. I wanted to check the file's timestamp and see if it is not getting updated(and did not want to get to watchdog etc. coz I just needed if a file stopped writing.)

try:
    if time.time()>(os.stat(filename).st_mtime+2):
        raise ValueError("Yikes! Spike")
except ValueError:
    with open('errors.log','a') as log:
        log.write('Spike occured at '+ time.strftime(
        "%H:%M:%S")+' on '+datetime.date.today().strftime('%d/%m/%Y')+'\n')
        log.close()
    restart_program()

This block runs every second. But this backfired and when the app closes for restarting it keeps closing every second and doesn't start again. I get the exception message logged every second. I tried increasing the time difference but that didn't help.

Next I tried

ftimestamp = os.stat(filename).st_mtime
try:
    if os.stat(filename).st_mtime>=ftimestamp:
        ftimestamp = time.time()
        print "ftimestamp updated and all is well"
    else:
        ftimestamp = os.stat(filename).st_mtime
        raise ValueError("Yikes! Spike!")
        print "file time is behind"
except ValueError:
    with open('errors.log','a') as log:
        log.write('Spike occured at '+ time.strftime(
        "%H:%M:%S")+' on '+datetime.date.today().strftime('%d/%m/%Y')+'\n')
        log.close()
    restart_program()

I tried updating the variable "ftimestamp" to current time "time.time()" because the next comparison happens only after one second and I want the file time to be higher than the previous time comparison. (The block runs every second through wx.CallLater function).

My program fails still... and I am not able to understand where I am going wrong... Someone please help! Or is there a way of simply checking if the file stopped writing?

Answer 1

We can try checking for a change in file size as a possible solution by doing the following:

import os
from time import sleep
# other imports

while True:
    file1 = os.stat('file.txt') # initial file size
    file1_size = file1.st_size
 
    # your script here that collects and writes data (increase file size)
    sleep(1)
    file2 = os.stat('file.txt') # updated file size
    file2_size = file2.st_size
    comp = file2_size - file1_size # compares sizes
    if comp == 0:
        restart_program()
    else:
        sleep(5)

You may need to adjust the sleep() function accordingly these are just estimates that I'm using since I can't test your actual code. In the end this is infinite loop that will keep running as long as you want the script to keep writing.

Another solution is to update your code to:

import os
import sys
from time import sleep
# other imports

while True:
    file1 = os.stat('file.txt') # initial file size
    file1_size = file1.st_size
 
    # your script here that collects and writes data (increase file size)
    sleep(1)
    file2 = os.stat('file.txt') # updated file size
    file2_size = file2.st_size
    comp = file2_size - file1_size # compares sizes
    if comp == 0:
        sys.exit
    else:
        sleep(5)

Then use a secondary program to run your script as such:

import os
from time import sleep, strftime

while True:
    print(strftime("%H:%M:%S"), "Starting"))
    system('main.py') # this is another infinite loop that will keep your script running
    print(strftime("%H:%M:%S"), "Crashed"))
    sleep(5)