How do you test a function using which retrieves data by urllib2?

Question

I am getting into testing in python and I asked myself how to test this method.

def get_response(self, url, params):
    encoded_params = urllib.urlencode(params)
    request = urllib2.Request(BASE_URL, headers=HEADERS)
    response = urllib2.urlopen(request, encoded_params)
    return response

Using doctest or unittest, how is this best achieved? I thought of passing get_response() a test url and some test parameters, which exists in real world and to check if response.read() returns the expected data. But somehow I feel, this is not the way it should be done. Any suggestions? I would appreciate suggestions how to handle such cases in context of texting.

Answer 1

This is a good opportunity to use a mock testing framework, such as minimock.

BASE_URL='http://google.com'
HEADERS = {"accept-language":"klingon"}
import urllib, urllib2
def get_response(self, url, params):
    r"""
    >>> from minimock import Mock
    >>> urllib2.Request = Mock('urllib2.Request')
    >>> urllib2.urlopen = Mock('urllib2.urlopen')
    >>> get_response(None, 'http://google.com', {'foo':'bar'})
    Called urllib2.Request(
        'http://google.com',
        headers={'accept-language': 'klingon'})
    Called urllib2.urlopen(None, 'foo=bar')
    >>> 
    """
    encoded_params = urllib.urlencode(params)
    request = urllib2.Request(BASE_URL, headers=HEADERS)
    response = urllib2.urlopen(request, encoded_params)
    return response

Do note that the unit is embedded in the docstring for the function under test, in doctest format.