Search code examples
pythonexceptionpython-requestspython-3.3python-mock

Can't catch mocked exception because it doesn't inherit BaseException


I'm working on a project that involves connecting to a remote server, waiting for a response, and then performing actions based on that response. We catch a couple of different exceptions, and behave differently depending on which exception is caught. For example:

def myMethod(address, timeout=20):
    try:
        response = requests.head(address, timeout=timeout)
    except requests.exceptions.Timeout:
        # do something special
    except requests.exceptions.ConnectionError:
        # do something special
    except requests.exceptions.HTTPError:
        # do something special
    else:
        if response.status_code != requests.codes.ok:
            # do something special
        return successfulConnection.SUCCESS

To test this, we've written a test like the following

class TestMyMethod(unittest.TestCase):

    def test_good_connection(self):
        config = {
            'head.return_value': type('MockResponse', (), {'status_code': requests.codes.ok}),
            'codes.ok': requests.codes.ok
        }
        with mock.patch('path.to.my.package.requests', **config):
            self.assertEqual(
                mypackage.myMethod('some_address',
                mypackage.successfulConnection.SUCCESS
            )

    def test_bad_connection(self):
        config = {
            'head.side_effect': requests.exceptions.ConnectionError,
            'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError
        }
        with mock.patch('path.to.my.package.requests', **config):
            self.assertEqual(
                mypackage.myMethod('some_address',
                mypackage.successfulConnection.FAILURE
            )

If I run the function directly, everything happens as expected. I even tested by adding raise requests.exceptions.ConnectionError to the try clause of the function. But when I run my unit tests, I get

ERROR: test_bad_connection (test.test_file.TestMyMethod)
----------------------------------------------------------------
Traceback (most recent call last):
  File "path/to/sourcefile", line ###, in myMethod
    respone = requests.head(address, timeout=timeout)
  File "path/to/unittest/mock", line 846, in __call__
    return _mock_self.mock_call(*args, **kwargs)
  File "path/to/unittest/mock", line 901, in _mock_call
    raise effect
my.package.requests.exceptions.ConnectionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Path/to/my/test", line ##, in test_bad_connection
    mypackage.myMethod('some_address',
  File "Path/to/package", line ##, in myMethod
    except requests.exceptions.ConnectionError:
TypeError: catching classes that do not inherit from BaseException is not allowed

I tried to change the exception I was patching in to BaseException and I got a more or less identical error.

I've read https://stackoverflow.com/a/18163759/3076272 already, so I think it must be a bad __del__ hook somewhere, but I'm not sure where to look for it or what I can even do in the mean time. I'm also relatively new to unittest.mock.patch() so it's very possible that I'm doing something wrong there as well.

This is a Fusion360 add-in so it is using Fusion 360's packaged version of Python 3.3 - as far as I know it's a vanilla version (i.e. they don't roll their own) but I'm not positive of that.


Solution

  • I could reproduce the error with a minimal example:

    foo.py:

    class MyError(Exception):
        pass
    
    class A:
        def inner(self):
            err = MyError("FOO")
            print(type(err))
            raise err
        def outer(self):
            try:
                self.inner()
            except MyError as err:
                print ("catched ", err)
            return "OK"
    

    Test without mocking :

    class FooTest(unittest.TestCase):
        def test_inner(self):
            a = foo.A()
            self.assertRaises(foo.MyError, a.inner)
        def test_outer(self):
            a = foo.A()
            self.assertEquals("OK", a.outer())
    

    Ok, all is fine, both test pass

    The problem comes with the mocks. As soon as the class MyError is mocked, the expect clause cannot catch anything and I get same error as the example from the question :

    class FooTest(unittest.TestCase):
        def test_inner(self):
            a = foo.A()
            self.assertRaises(foo.MyError, a.inner)
        def test_outer(self):
            with unittest.mock.patch('foo.MyError'):
                a = exc2.A()
                self.assertEquals("OK", a.outer())
    

    Immediately gives :

    ERROR: test_outer (__main__.FooTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "...\foo.py", line 11, in outer
        self.inner()
      File "...\foo.py", line 8, in inner
        raise err
    TypeError: exceptions must derive from BaseException
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<pyshell#78>", line 8, in test_outer
      File "...\foo.py", line 12, in outer
        except MyError as err:
    TypeError: catching classes that do not inherit from BaseException is not allowed
    

    Here I get a first TypeErrorthat you did not have, because I am raising a mock while you forced a true exception with 'requests.exceptions.ConnectionError': requests.exceptions.ConnectionError in config. But the problem remains that the except clause tries to catch a mock.

    TL/DR: as you mock the full requests package, the except requests.exceptions.ConnectionError clause tries to catch a mock. As the mock is not really a BaseException, it causes the error.

    The only solution I can imagine is not to mock the full requests but only the parts that are not exceptions. I must admit I could not find how to say to mock mock everything except this but in your example, you only need to patch requests.head. So I think that this should work :

    def test_bad_connection(self):
        with mock.patch('path.to.my.package.requests.head',
                        side_effect=requests.exceptions.ConnectionError):
            self.assertEqual(
                mypackage.myMethod('some_address',
                mypackage.successfulConnection.FAILURE
            )
    

    That is : only patch the head method with the exception as side effect.