i am developing a machine learning algorithm to predict temperature in my city. I am using wunderground API in order to obtain the data. So, in this piece of code where I execute a request to the API:
def extract_weather_data(url, api_key, target_date, days):
records = []
for _ in range(days):
request = BASE_URL.format(API_KEY, target_date.strftime('%Y%m%d'))
response = requests.get(request)
print request
if response.status_code == 200:
data = response.json()['history']['dailysummary'][0]
records.append(DailySummary(
date=target_date,
meantempm=data['meantempm'],
meandewptm=data['meandewptm'],
meanpressurem=data['meanpressurem'],
maxhumidity=data['maxhumidity'],
minhumidity=data['minhumidity'],
maxtempm=data['maxtempm'],
mintempm=data['mintempm'],
maxdewptm=data['maxdewptm'],
mindewptm=data['mindewptm'],
maxpressurem=data['maxpressurem'],
minpressurem=data['minpressurem'],
precipm=data['precipm']))
time.sleep(6)
target_date += timedelta(days=1)
return records
records = extract_weather_data(BASE_URL, API_KEY, target_date, 100)
I obtain this error after 3-4 request:
Traceback (most recent call last):
File "data.py", line 45, in <module>
records = extract_weather_data(BASE_URL, API_KEY, target_date, 100)
File "data.py", line 26, in extract_weather_data
data = response.json()['history']['dailysummary'][0]
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 892, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name enclosed in double quotes: line 11 column 1 (char 155)
How can i fix this?
Thanks you a lot!
Golden rule when it comes to HTTP requests (API or not and whatever the expected response's content type) is that things can go wrong in many ways and at any time and that they WILL go wrong one day or another in the most unexpected way, so if you hope to have a decenty robust client program, you have to be prepared to handle just any possible outcome.
Most often, this actually means wrapping requests calls in try/except handlers plus testing the response's status code, content-type and effective content (instead of blindly assuming you got what you asked for), and depending on the exception / unexpected response value decide whether it's worth retrying the request a couple times (with increasing delays between the retries) before giving up (some error conditions can be temporary) or just give up immediatly. In all cases, you also want to make sure you let the user know what went wrong, with as much informations as possible (request, response if you got one, exception and full traceback else) when you decide to give up.