How to properly manage API call limits in a Python script?

I am writing a script that iterates over the elements of a list and for each one calls an api. Something like:

myList=['a','b',...] #200k items 

result_dict={}

for item in myList:
    result=myapi.myfunction(item) #I think this way is slower since it has to check credentials every iteration
    result_dict[item]=result

or, alternatively (maybe faster, I haven't checked)

myList=['a','b',...] #200k items 

result_dict={}

results=myapi.myfunction(myList) #I can pass either an item or a list

for id_, result in results:
     result_dict[id_]=result

I have a limit of 20.000 calls per day, after which I get billed for every extra request, and I am assuming that I will be able to fulfill all the 20k calls before the end of the day. Therefore, I need to think of some way to control it.

I thought about counting the calls made and, when I reach the 20k, I put the code on stand-by until the following midnight. Does it make sense? How can I implement it, for example through the time.sleep() function? I have no problem in leaving the script running, since I am working on an AWS remote server.

Can you suggest anything better?

Solution

You have two options: spread the calls over the whole day (20000 / 24 calls per hour) or stop calling until midnight once you reach 20000 calls.

Just add a counter to the API calls, and a conditional to check wether you reached 20k calls or not.

If 20000 calls were reached, you can time.sleep() t seconds until midnight.

You can compute the number of seconds until midnight by using datetime:

import datetime
import time
now = datetime.datetime.now()
midnight = datetime.datetime.now() + datetime.timedelta(days=1)
midnight = midnight.replace(hour=0, minute=0, second=0)
seconds_until_midnight = (midnight - now).total_seconds()
time.sleep(seconds_until_midnight)