Nose: generator for TestCase-based classes

Question

I want to create a generator for variations of a TestCase-derived class.

What I tried is this:

import unittest

def create_class(param):
    class Test(unittest.TestCase):
        def setUp(self):
            pass

        def test_fail(self):
            assert False
    return Test

def test_basic():
    for i in range(5):
        yield create_class(i)

What I get is this:

======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/nose/case.py", line 268, in setUp
    try_run(self.test, names)
  File "/usr/lib/python3.3/site-packages/nose/util.py", line 478, in try_run
    return func()
TypeError: setUp() missing 1 required positional argument: 'self'

Yielding instances instead of classes (yield create_class(i)()) leaves me with this error:

======================================================================
ERROR: test_1.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/usr/lib/python3.3/unittest/case.py", line 492, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.3/unittest/case.py", line 423, in run
    testMethod = getattr(self, self._testMethodName)
AttributeError: 'Test' object has no attribute 'runTest'

Any ideas?

Answer 1

When instantiating a TestCase you should pass the method name of the test:

yield create_class(i)('test_fail')

Otherwise the name defaults to runTest(and thus the last error you got).

Also note that there is a strange interaction between test generators and TestCase. With the following code:

import unittest

def create_class(param):
    class Test(unittest.TestCase):
        def setUp(self):
            pass

        def test_fail(self):
            print('executed')
            assert False
            print('after assert')

    return Test

def test_basic():
    for i in range(5):
        yield create_class(i)('test_fail')

I obtain this output:

$ nosetests -s
executed
.executed
.executed
.executed
.executed
.
----------------------------------------------------------------------
Ran 5 tests in 0.004s

OK

As you can see the test does not fail, even though the assert works. This is probably due to the fact that TestCase handles the AssertionError but nose does not expect this to be handled and thus it cannot see that the test failed.

This can be seen from the documentation of TestCase.run:

Run the test, collecting the result into the test result object passed as result. If result is omitted or None, a temporary result object is created (by calling the defaultTestResult() method) and used. The result object is not returned to run()‘s caller.

The same effect may be had by simply calling the TestCase instance.

So, nose doesn't see that the objected yielded by the generator is a TestCase which should be handled in a special manner, it simply expects a callable. The TestCase is run, but the result is put into a temporary object that is lost, and this eats all test failures that happen inside the tests. Hence yielding TestCasees simply doesn't work.