Best-practice: automated web API testing

Question

I've written a program in Python, which works with two distinct API to get the data from two different services (CKAN and MediaWiki). In particular, there is a class Resource, which requests the data from the above mentioned services and process it.

At some point I've come to conclusion, that there is a need for tests for my app. And the problem is that all examples I've found on web and in books do not deal with such cases.

For example, inside Resource class I've got a method:

def load_from_ckan(self):
    """
        Get the resource
        specified by self.id
        from config.ckan_api_url 
    """
    data = json.dumps({'id': self.id})
    headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
    url = config.ckan_api_url + '/action/resource_show'
    r = requests.post(url, timeout=config.ckan_request_timeout, data=data, headers=headers)
    assert r.ok, r
    resource = json.loads(r.content)
    resource = resource["result"]
    for key in resource:
        setattr(self, key, resource[key])

The load_from_ckan method get the data about resource from the CKAN API and assign it to the object. It is simple, but...

My question is: how to test the methods like this? OR What should I test here?

I thought about the possibility to pickle (save) results to HDD. Then I could load it in the test and compare with the object initialized with load_from_ckan(). But CKAN is community-driven platform and such behavior of such tests will be unpredictable.

If there exist any books on philosophy of automated tests (like what to test, what not to test, how to make tests meaningful etc.), please, give me a link to it.

Answer 1

With any testing, the key question really is - what could go wrong?

In your case, it looks like the three risks are:

• The web API in question could stop working. But you check for this already, with assert r.ok.
• You, or someone else, could make a mistaken change to the code in future (e.g. mistyping a variable) which breaks it.
• The API could change, so that it no longer returns the fields or the format you need.

It feels like you could write a fairly simple test for the latter two, depending on what data from this API you actually rely on: for example, if you're expecting the JSON to have a field called "temperature" which is a floating-point Celsius number, you could write a test which calls your function, then checks that self.temperature is an instance of 'float' and is within a sensible range of values (-30 to 50?). That should leave you confident that both the API and your function are working as designed.