How to create a unit test for a function that uses Python's http.client library using pytest and mocks?

Question

How do I write a mock test for the following function using pytest?

import http.client

def get_response(req_type, host, sub_domain, payload=None, headers=None,
                 body=None):

    conn = http.client.HTTPSConnection(host)
    conn.request(req_type, sub_domain, headers=headers, body=payload)
    response = conn.getresponse()

    if response.status != 200:
        raise Exception('invalid http status ' + str(response.status)
                        + ',detail body:' + response.read().decode("utf-8"))

    data = response.read().decode("utf-8")
    conn.close()

    return data

By using the pytest_mock library and referring to the code here, I was able to unit tests for the other functions which use get_response() to perform some actions. So it's fine if we need to use even pytest_mock library to perform the unittest for get_response.

Having said that, solutions that I have seen so far are geared towards requests and unittest libraries.

I would like to avoid creating a http server or a Flask server for this mock-based unit-testing.

The pytest documentation seems to sugguest that I need a patch for http.client.HTTPSConnection, conn.request() & conn.getresponse() to unit test get_response().

Is that the case?

A minimal working example would be helpful.

Answer 1

Here is a minimal working example (assume your method is placed in http_call.py):

from http_call import get_response


def test_get_response(mocker):
  # mocked dependencies
  mock_HTTPSConnection = mocker.MagicMock(name='HTTPSConnection')
  mocker.patch('http_call.http.client.HTTPSConnection', new=mock_HTTPSConnection)
  mock_HTTPSConnection.return_value.getresponse.return_value.status = 200
  mock_HTTPSConnection.return_value.getresponse.return_value.read.return_value.decode.return_value = "some html goes here"

  # act
  data = get_response("GET", "www.google.com", '/', headers={})

  # assert
  assert "some html goes here" == data

The example explained:

1. You mock the "root" of your https connection, which is the HTTPSConnection class. Then you have to ensure your response is 200 and return some data you can assert.
2. Then you call the function and get the output
3. You can then assert the output with the predefined hardcoded value you had mocked

A few additional things you can do in this test function:

Add asserts, using my pytest-mock-generator library fixture like so:

def test_get_response(mocker, mg):
    # mocked dependencies
    mock_HTTPSConnection = mocker.MagicMock(name='HTTPSConnection')
    mocker.patch('http_call.http.client.HTTPSConnection', new=mock_HTTPSConnection)
    mock_HTTPSConnection.return_value.getresponse.return_value.status = 200
    mock_HTTPSConnection.return_value.getresponse.return_value.read.return_value.decode.return_value = "some html goes here"

    # act
    data = get_response("GET", "www.google.com", '/', headers={})

    # assert
    assert "some html goes here" == data

    # this code generates extra asserts
    mg.generate_asserts(mock_HTTPSConnection)

This would generate the output (printed to the console and copied to your clipboard):

assert 1 == mock_HTTPSConnection.call_count
mock_HTTPSConnection.assert_called_once_with('www.google.com')
mock_HTTPSConnection.return_value.request.assert_called_once_with('GET', '/', body=None, headers={})
mock_HTTPSConnection.return_value.getresponse.assert_called_once_with()
mock_HTTPSConnection.return_value.getresponse.return_value.read.assert_called_once_with()
mock_HTTPSConnection.return_value.getresponse.return_value.read.return_value.decode.assert_called_once_with('utf-8')
mock_HTTPSConnection.return_value.close.assert_called_once_with()

These extra asserts can help you ensure that you're calling the functions with the right parameters.

Another thing my library can do is to generate the initial mocks, by analyzing your code, like so:

def test_get_response(mocker, mg):
    mg.generate_uut_mocks(get_response)

You would get this output:

# mocked dependencies
mock_HTTPSConnection = mocker.MagicMock(name='HTTPSConnection')
mocker.patch('http_call.http.client.HTTPSConnection', new=mock_HTTPSConnection)
mock_Exception = mocker.MagicMock(name='Exception')
mocker.patch('http_call.Exception', new=mock_Exception)
mock_str = mocker.MagicMock(name='str')
mocker.patch('http_call.str', new=mock_str)

Obviously no need to mock Exception and str, so you can drop those suggestions.