JSON list/dictionary parsing from API

Question

I have developed a small library and am interested in making it easier on users to retrieve data from the JSON lists/dictionaries that are returned. I have created functions that handle the calls using requests. Now suppose I call this function and pass in a few parameters:

precip = precipitation_obs(stid='kfnl', start='201504261800', end='201504271200', units='precip|in')

This will return the following JSON:

{ 'STATION': [ { 'ELEVATION': '5016',
                 'ID': '192',
                 'LATITUDE': '40.45',
                 'LONGITUDE': '-105.01667',
                 'MNET_ID': '1',
                 'NAME': 'Fort Collins/Loveland, Fort Collins-Loveland '
                         'Municipal Airport',
                 'OBSERVATIONS': { 'count_1': 6,
                                   'ob_end_time_1': '2015-04-27T00:55:00Z',
                                   'ob_start_time_1': '2015-04-26T18:55:00Z',
                                   'total_precip_value_1': 0.13,
                                   'vids case4': ['39', '51', '40', '52']},
                 'STATE': 'CO',
                 'STATUS': 'ACTIVE',
                 'STID': 'KFNL',
                 'TIMEZONE': 'US/Mountain'}],
  'SUMMARY': { 'METADATA_RESPONSE_TIME': '5.22613525391 ms',
               'NUMBER_OF_OBJECTS': 1,
               'RESPONSE_CODE': 1,
               'RESPONSE_MESSAGE': 'OK',
               'TOTAL_TIME': '57.6429367065 ms'}}    

Now, I want the user to be able to just drill down through the dictionary, but STATION is a list and requires me to do the following:

output =  precip['STATION'][0]['OBSERVATIONS']['ob_start_time_1'] 
print(output)
# returns 2015-04-26T18:55:00Z

where I have to include the [0] to avoid:

TypeError: list indices must be integers, not str

Is there anyway around this? Adding that [0] in there really jacks things up so to say. Or even having to specify ['STATION'] every time is a bit of a nuisance. Should I use the simpleJSON module to help here? Any tips on making this a bit easier would be great, thanks!

Answer 1

Adding that [0] in there really jacks things up so to say. Or even having to specify ['STATION'] every time is a bit of a nuisance.

So just store precip['STATION'][0] in a variable:

>>> precip0 = precip['STATION'][0]

And now, you can use it repeatedly:

>>> precip0['OBSERVATIONS']['ob_start_time_1']
2015-04-26T18:55:00Z

If you know that the API is always going to return exactly one station, and you're never going to want anything other than the data for that station, you can put that in your wrapper function:

def precipitation_obs(stid, start, end, units):
    # your existing code, which assigns something to result
    return result['STATION'][0]

---

If you're worried about "efficiency" here, don't worry. First, this isn't making a copy of anything, it's just making another reference to the same object that already exists—it takes under a microsecond, and wastes about 8 bytes. In fact, it saves you memory, because if you're not storing the whole dict, just the sub-dict, Python can garbage-collect the rest of the structure. And, more importantly, that kind of micro-optimization isn't worth worrying about in the first place until (1) your code is working and (2) you know it's a bottleneck.

---

Should I use the simpleJSON module to help here?

Why would you think that would help? As its readme says:

simplejson is the externally maintained development version of the json library included with Python 2.6 and Python 3.0, but maintains backwards compatibility with Python 2.5.

In other words, it's either the same code you've already got in the stdlib, or an older version of that code.

Sometimes a different library, like ijson, can help—e.g., if the JSON structures are so big that you can't parse the whole thing into memory, or so complex that it's easier to describe what you want upside-down in SAX style. But that's not relevant here.