How can I pause part of a function for some period of time?

Question

I have a code like this:

def1():
    a = requests.get(url)
    ...

def2():
    something that calls def1()

def3():
    something that calls def2()

def4():
    something that calls def2()

defN():
    something that calls def(2)

And I want to schedule all these function to execute once every a fixed period of time X. To do that I used schedule module with no problems for the first functions.

schedule.every.hour.do(def1)
schedule.every.hour.do(def2)
...

But I would like defN to collect the data from the url (possible because it calls def2 that calls def1), but unlike the other functions that use the url information immediately, I want defN to hold the url request data in a variable for the same fixed period of time X, and then proceed with its code.

The aim of the function is to collect every hour the url information of the previous hour and then do operations between previous hour data and current hour data. Of course in the first hour when the program works, defN doesn't return anything. Is there a way to pause the first part of the function for one hour and then schedule it every hour?

I tried to use time.sleep() inside defN but it stopped also the other functions for two hours. I learned Thread module but I never get it to work properly. I put this code in every other function except defN:

t = threading.Thread(target=defN)
t.start()

But it didn't work. How can I solve this?

Answer 1

There a few ways you can achieve this goal. I’m going to provide you one way that works with your code with minimal changes, but it’s not the optimal way because you didn’t ask for a better or optimal approach.

First, note that you already have a natural 1 hour delay using that scheduler. Therefore, defN does not need to sleep or wait or block.

You could define a global variable defN_prev_data. Initially assign None to it. Then you defN function can look like this:

# create global var and unit to None
defN_prev_data = None

def defN()
    # specify global here because this function will modify it
    global defN_prev_data
    new_data = something that calls def2()
    if defN_prev_data is not None:
        process defN_prev_data
    defN_prev_data = new_data


Schedule defN()

Just make sure the global variable is declared in your script or module and initialized to None before defN is scheduled or called.

Also, if you ever stop and restart the scheduled tasks within the same proceeds context, you will need to reset defN_prev_data back to None or you’ll process old data, which might not be what you want.