I am trying to learn the python api for news sources so I copied the python notebook code from here: source
When I run the code, exactly as posted except for the api key, I see the error below (the quote limit is 10). It appears even when I expend to 30 seconds from 6. I am not sure why this error appears because it did not appear for the code creator (their entire response was posted). Sorry for my dumb questions, I am a novice.
This is my error message:
Working on ['2015-01-01', '00:00:00']...
Working on ['2015-02-01', '00:00:00']...
Working on ['2015-03-01', '00:00:00']...
Working on ['2015-04-01', '00:00:00']...
Working on ['2015-05-01', '00:00:00']...
Working on ['2015-06-01', '00:00:00']...
Working on ['2015-07-01', '00:00:00']...
Working on ['2015-08-01', '00:00:00']...
Working on ['2015-09-01', '00:00:00']...
Working on ['2015-10-01', '00:00:00']...
Working on ['2015-11-01', '00:00:00']... <-- Always stops here
in parse_response
{'fault': {'faultstring': 'Rate limit quota violation. Quota limit exceeded. 'detail': {'errorcode': 'policies.ratelimit.QuotaViolation'}}}
{'headline': [], 'date': [], 'doc_type': [], 'material_type': [], 'section': [], 'keywords': []}
url comparison:
works at nyt api: https://api.nytimes.com/svc/archive/v1/2019/1.json?api-key=YOUR_API_KEY
code generated url states access denied even though I have access to NYT archive:
https://api.nytimes.com/svc/archive/v1/2016-01-01/00:00:00.json?api-key=YOUR_API_KEY
'''fixed url new error message when running from python'''
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
--NotebookApp.iopub_data_rate_limit.
Current values: NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec) NotebookApp.rate_limit_window=3.0 (secs)
but works in browser: https://api.nytimes.com/svc/archive/v1/2016/1.json?api-key=YOUR_API_KEY
I saw others asking this question with google but this is nytimes.
Thanx
Ensure that you wait regardless of whether an Exception is raised:
def send_request(date):
'''Sends a request to the NYT Archive API for given date.'''
base_url = 'https://api.nytimes.com/svc/archive/v1/'
url = base_url + '/' + date[0] + '/' + date[1] + '.json?api-key=' + YOUR_API_KEY
try:
response = requests.get(url, verify=False).json()
except Exception:
return None
finally:
time.sleep(6)
return response
The finally clause is the last block of code executed before the try or except blocks complete. This means that the finally block will execute before any return statement in the try, except or else (not applicable here) blocks. This can be very confusing. I suggest putting a single return statement at the end of the function like this:
def send_request(date):
'''Sends a request to the NYT Archive API for given date.'''
base_url = 'https://api.nytimes.com/svc/archive/v1/'
url = base_url + '/' + date[0] + '/' + date[1] + '.json?api-key=' + YOUR_API_KEY
try:
response = requests.get(url, verify=False).json()
except Exception:
response = None
finally:
time.sleep(6)
return response
Both will have the same result: sleep six seconds regardless of the response and return either a Response or None, but I think the second is easier to understand. Now, you still need to figure out why the Exceptions are occurring and fix that. I suggest adding some print() debugging statements in the except block to figure out exactly what's occurring.
Another option would be to assume that the reader fully understands Python's try-except-else-finally behavior:
def send_request(date):
'''Sends a request to the NYT Archive API for given date.'''
base_url = 'https://api.nytimes.com/svc/archive/v1/'
url = base_url + '/' + date[0] + '/' + date[1] + '.json?api-key=' + YOUR_API_KEY
try:
return requests.get(url, verify=False).json()
except Exception:
return None
finally:
time.sleep(6)
This article does an excellent job of explaining the behavior of Python's try-except-else-finally behavior.