Search code examples
pythonpython-unittest

Python unittest: make AssertionError an error instead of a failure


Python 2.7. The unittest doc says:

To make migrating existing test suites easier, unittest supports tests raising AssertionError to indicate test failure. However, it is recommended that you use the explicit TestCase.fail*() and TestCase.assert*() methods instead, as future versions of unittest may treat AssertionError differently.

I used quite a few assert statements inside the tested code, but failure of those assertions should really be a test error (i.e. "code did not run properly with those inputs") rather than a failure (i.e. "code gave incorrect output").

I can see the following possible solutions:

  1. Rewrite the tested code to throw better-typed exceptions
  2. Wrap everything inside test methods except the test assertions themselves (self.assertSomething(...)) in try...except AssertionError: raise SomeOtherException blocks
  3. Change unittest's behavior so that it considers those errors rather than failures.

Option 1 would take quite some time, and option 2 feels hacky; option 3 would be the best for me, but is it available? (In case it matters: no, I cannot switch to Python 3.) I do not see anything online, but it is hard to use specific keywords.

MWE:

import unittest


def add_one_to_int(a):
    assert isinstance(a, int)
    return a + 1


class TestAddOne(unittest.TestCase):
    def test_one_plus_one_is_three(self):
        # This tests fails with
        #   AssertionError: 2 != 3
        # which is fine
        self.assertEqual(add_one_to_int(1), 3)  

    def test_add_one_to_str(self):
        # This tests fails with
        #   AssertionError
        # when I would rather have it an error
        add_one_to_int('some string')

if __name__ == '__main__':
    unittest.main(verbosity=2)  # 2 failures instead of 1 failure, 1 error

Solution

  • I think that option 3 can be achieved through class attribute "failureException", as defined also in unittest docs for Python 2.7:

    failureException: This class attribute gives the exception raised by the test method. If a test framework needs to use a specialized exception, possibly to carry additional information, it must subclass this exception in order to “play fair” with the framework. The initial value of this attribute is AssertionError.

    So for example:

    import unittest
    
    class MyException(Exception): 
        pass
    
    class MyUnitTest(unittest.TestCase):
        failureException = MyException
    
    def add_one_to_int(a):
        assert isinstance(a, int)
        return a + 1
    
    
    class TestAddOne(MyUnitTest):  # <--------- See above
        def test_one_plus_one_is_three(self):
            # This tests fails with
            #   AssertionError: 2 != 3
            # which is fine
            self.assertEqual(add_one_to_int(1), 3)  
    
        def test_add_one_to_str(self):
            # This tests fails with
            #   AssertionError
            # when I would rather have it an error
            add_one_to_int('some string')
    
    if __name__ == '__main__':
        unittest.main(verbosity=2)  # ------> FAILED (failures=1, errors=1)