Python threading Timer not filling output file

Question

I am using the module threading class Timer. I want to set up a process that will be repeated every day, generating an output json file every time is launched (day).

The problem is that it generates a file that is NOT filled until the whole process finishes (so if it has to run for an entire year, I shall wait all the year).

Here the code.

from processing import *
import sys
from datetime import datetime, timedelta
from threading import Timer

if __name__ == "__main__":
    '''
    '''
    #Function that generates a json file, that (works), but only is filled if Timer is NOT runing
    geojson_gen(sys.argv[1],
                sys.argv[2],
                sys.argv[3],
                out_filename = 'test_country'
                )

    for rep in range(10000):
        #Get the number of seconds for the next time Timer will launch the func. geojson_gen(). Here some code to get the number of seconds in which it shall be launched the timer
        next_date = (launch_date - start_date).seconds

        t = Timer(next_date , geojson_gen(sys.argv[1],
                                    sys.argv[2],
                                    sys.argv[3],
                                    out_filename = 'test_country'
                                    )
                 )
        t.start()

So all the script runs properly and if I comment all the Timer part, I get the json file. But when I start the process to run everyday, it generates empty json files (not filled).

What is wrong? How to get the json filled after the function geojson_gen() finished (and not after the whole Timer process)?

Thanks a lot!

Answer 1

With the code given I don't know why empty files are generated, but starting lots of Timers in parallel is not a good solution for your goal. Because what you want is not a parallel process, but a repeated, sequential one.

A more efficient way is to create a time-based loop. This example shows the basic idea:

import time

#set the wait time (a day in seconds)
waitTime = 60*60*24 

# set initial value
startTime = time.time()

# runs forever
while True:
    # check how much time has passed
    timeDiff = time.time() - startTime
    # if a day has passed, generate a json and update starttime
    if timeDiff > waitTime:
        geojson_gen()
        startTime = time.time()

This means you have to leave the script running though. An even cleaner solution is to only use the script that creates the json, then schedule in your OS to run that script daily. See this answer for a Windows example.