How can I use unittest.mock to remove side effects from code?

Question

I have a function with several points of failure:

def setup_foo(creds):
    """
    Creates a foo instance with which we can leverage the Foo virtualization
    platform.

    :param creds: A dictionary containing the authorization url, username,
                  password, and version associated with the Foo
                  cluster.
    :type creds:  dict
    """

    try:
        foo = Foo(version=creds['VERSION'],
                  username=creds['USERNAME'],
                  password=creds['PASSWORD'],
                  auth_url=creds['AUTH_URL'])

        foo.authenticate()
        return foo
    except (OSError, NotFound, ClientException) as e:
        raise UnreachableEndpoint("Couldn't find auth_url {0}".format(creds['AUTH_URL']))
    except Unauthorized as e:
        raise UnauthorizedUser("Wrong username or password.")
    except UnsupportedVersion as e:
        raise Unsupported("We only support Foo API with major version 2")

and I'd like to test that all the relevant exceptions are caught (albeit not handled well currently).

I have an initial test case that passes:

def test_setup_foo_failing_auth_url_endpoint_does_not_exist(self):
    dummy_creds = {
        'AUTH_URL' : 'http://bogus.example.com/v2.0',
        'USERNAME' : '', #intentionally blank.
        'PASSWORD' : '', #intentionally blank.
        'VERSION'  : 2 
    }
    with self.assertRaises(UnreachableEndpoint):
        foo = osu.setup_foo(dummy_creds)

but how can I make my test framework believe that the AUTH_URL is actually a valid/reachable URL?

I've created a mock class for Foo:

class MockFoo(Foo):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

and my thought is mock the call to setup_foo and remove the side effect of raising an UnreachableEndpoint exception. I know how to add side-effects to a Mock with unittest.mock, but how can I remove them?

Answer 1

Assuming your exceptions are being raised from foo.authenticate(), what you want to realize here is that it does not necessarily matter whether the data is in fact really valid in your tests. What you are trying to say really is this:

When this external method raises with something, my code should behave accordingly based on that something.

So, with that in mind, what you want to do is have different test methods where you pass what should be valid data, and have your code react accordingly. The data itself does not matter, but it provides a documented way of showing how the code should behave with data that is passed in that way.

Ultimately, you should not care how the nova client handles the data you give it (nova client is tested, and you should not care about it). What you care about is what it spits back at you and how you want to handle it, regardless of what you gave it.

In other words, for the sake of your tests, you can actually pass a dummy url as:

"this_is_a_dummy_url_that_works"

For the sake of your tests, you can let that pass, because in your mock, you will raise accordingly.

For example. What you should be doing here is actually mocking out Client from novaclient. With that mock in hand, you can now manipulate whatever call within novaclient so you can properly test your code.

This actually brings us to the root of your problem. Your first exception is catching the following:

except (OSError, NotFound, ClientException)

The problem here, is that you are now catching ClientException. Almost every exception in novaclient inherits from ClientException, so no matter what you try to test beyond that exception line, you will never reach those exceptions. You have two options here. Catch ClientException, and just raise a custom exception, or, remote ClientException, and be more explicit (like you already are).

So, let us go with removing ClientException and set up our example accordingly.

So, in your real code, you should be now setting your first exception line as:

except (OSError, NotFound) as e:

Furthermore, the next problem you have is that you are not mocking properly. You are supposed to mock with respect to where you are testing. So, if your setup_nova method is in a module called your_nova_module. It is with respect to that, that you are supposed to mock. The example below illustrates all this.

@patch("your_nova_module.Client", return_value=Mock())
def test_setup_nova_failing_unauthorized_user(self, mock_client):
    dummy_creds = {
        'AUTH_URL': 'this_url_is_valid',
        'USERNAME': 'my_bad_user. this should fail',
        'PASSWORD': 'bad_pass_but_it_does_not_matter_what_this_is',
        'VERSION': '2.1',
        'PROJECT_ID': 'does_not_matter'
    }

    mock_nova_client = mock_client.return_value
    mock_nova_client.authenticate.side_effect = Unauthorized(401)

    with self.assertRaises(UnauthorizedUser):
        setup_nova(dummy_creds)

So, the main idea with the example above, is that it does not matter what data you are passing. What really matters is that you are wanting to know how your code will react when an external method raises.

So, our goal here is to actually raise something that will get your second exception handler to be tested: Unauthorized

This code was tested against the code you posted in your question. The only modifications were made were with module names to reflect my environment.