How to avoid KeyError from missing dict keys?

Question

I'm running a script that makes GET requests in a loop. I test for a successful response, then convert the response to a json object and do stuff. My code looks like this:

response = requests.get(url=url)
if response.status_code == 200:
    response_json = response.json()
    <do stuff>
else:
    <handle error>

Sometimes I get a successful response, but for whatever reason the dict comes back with some missing data and my script breaks from a KeyError. I want to make a bulletproof test to avoid errors and retrieve the value of 'high' and 'low' for response_json['Data']['Data'][1]. The structure of the dict is:

{'Data': {'Aggregated': False,
          'Data': [{'close': 0.8062,
                    'conversionType': 'multiply',
                    'high': 0.8084,
                    'low': 0.788,
                    'open': 0.8102,
                    'time': 1428465600,
                    'volumefrom': 145.38,
                    'volumeto': 117.2},
                   {'close': 0.8,
                    'conversionType': 'multiply',
                    'high': 0.8101,
                    'low': 0.8,
                    'open': 0.8062,
                    'time': 1428469200,
                    'volumefrom': 262.39,
                    'volumeto': 209.92}],
          'TimeFrom': 1428465600,
          'TimeTo': 1428469200},
 'HasWarning': False,
 'Message': '',
 'RateLimit': {},
 'Response': 'Success',
 'Type': 100}

And my current best attempt at a test is:

if 'Data' in response_json:
    if 'Data' in 'Data':
        if len(response_json['Data']['Data']) > 1:
            if 'low' and 'high' in response_json['Data']['Data'][1]:
                if 'low' != None and 'high' != None:
                    <do stuff>    
else:
    print("Error:", url)

I believe this covers all bases, but it seems unwieldy. Is there a better way to test for the presence of keys and valid values in this situation, and/or to keep my script running without breaking?

Also wondering if I need an else statement after every nested conditional test, or if Python will default to the else at the bottom if one of the conditions comes back False?

Answer 1

I could not find any XPath type way to find a key in json so the stacked if statements are needed.

You need to fix your syntax to get correct values.

Try this code:

response_json = {'Data': {'Aggregated': False,
          'Data': [{'close': 0.8062,
                    'conversionType': 'multiply',
                    'high': 0.8084,
                    'low': 0.788,
                    'open': 0.8102,
                    'time': 1428465600,
                    'volumefrom': 145.38,
                    'volumeto': 117.2},
                   {'close': 0.8,
                    'conversionType': 'multiply',
                    'high': 0.8101,
                    'low': 0.8,
                    'open': 0.8062,
                    'time': 1428469200,
                    'volumefrom': 262.39,
                    'volumeto': 209.92}],
          'TimeFrom': 1428465600,
          'TimeTo': 1428469200},
 'HasWarning': False,
 'Message': '',
 'RateLimit': {},
 'Response': 'Success',
 'Type': 100}


low = high = None  # default values
if 'Data' in response_json.keys():
    if 'Data' in response_json['Data'].keys():
        if len(response_json['Data']['Data']) > 1:
            if 'low' in response_json['Data']['Data'][1]:
               if 'high' in response_json['Data']['Data'][1]:
                  if response_json['Data']['Data'][1]['low'] and response_json['Data']['Data'][1]['high']:
                        low  = response_json['Data']['Data'][1]['low']
                        high = response_json['Data']['Data'][1]['high']

if low and high:   # actually only need to check one                      
   print('<do stuff>', 'low', response_json['Data']['Data'][1]['low'])
   print('<do stuff>', 'high', response_json['Data']['Data'][1]['high'])
else:
   print("High\Low not found")