Search code examples
pythonpython-3.xpython-multithreadingconcurrent.futuressched

Schedule an iterative function every x seconds without drifting


Complete newbie here so bare with me. I've got a number of devices that report status updates to a singular location, and as more sites have been added, drift with time.sleep(x) is becoming more noticeable, and with as many sites connected now it has completely doubles the sleep time between iterations.

import time


...
def client_list():
    sites=pandas.read_csv('sites')
    return sites['Site']


def logs(site):
    time.sleep(x)
    if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
        stamp = time.strftime('%Y-%m-%d,%H:%M:%S')
        log = open(f"{site}/log", 'a')
        log.write(f",{stamp},{site},hit\n")
        log.close()
        os.remove(f"{site}/target/hit")
    else:
        stamp = time.strftime('%Y-%m-%d,%H:%M:%S')
        log = open(f"{site}/log", 'a')
        log.write(f",{stamp},{site},miss\n")
        log.close()
...


if __name__ == '__main__':
    while True:
        try:
            client_list()
            with concurrent.futures.ThreadPoolExecutor() as executor:
                executor.map(logs, client_list())
...

I did try adding calculations for drift with this:

from datetime import datetime, timedelta


def logs(site):
    first_called=datetime.now()
    num_calls=1
    drift=timedelta()
    time_period=timedelta(seconds=5)
    while 1:
        time.sleep(n-drift.microseconds/1000000.0)
        current_time = datetime.now()
        num_calls += 1
        difference = current_time - first_called
        drift = difference - time_period* num_calls
        if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
...

It ends up with a duplicate entries in the log, and the process still drifts. Is there a better way to schedule the function to run every x seconds and account for the drift in start times?


Solution

  • So I managed to find another route that doesn't drift. The other method still drifted over time. By capturing the current time and seeing if it is divisible by x (5 in the example below) I was able to keep the time from deviating.

    def timer(t1,t2)
        return True if t1 % t2 == 0 else False
    
    def logs(site):
      while 1:
        try:
          if timer(round(time.time(), 0), 5.0):
             if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
                # do something that takes a while
                time.sleep(1) ''' this kept it from running again immediately if the process was shorter than 1 second. '''
    ...