nose tests with custom plugin never enters teardown on assertion errors

Question

I have a selenium test suite run by nose. We have written a custom plugin that pushes info to our db in afterTest() but this returns nothing, so it should not interfere with any of nose's normal functionality.

If a test has issues due to timeout or some other browser error, tests enter teardown() just fine, and nose registers an error.

But if a test fails on assertion error, nose registers these as errors (not failures) and teardown() is never triggered, leaving me with lots of orphaned browsers since self.driver.quit() lives in teardown().

For example, we have a test that performs function on our webpage which has a loading indicator. We want to verify that the indicator disappears in a timely fashion, so we've written this function into our testing library.

def Check_for_global_indicator_doesnt_exist(unit_test):
     global_indicator_doesnt_exists=unit_test.wait_element_doesnt_exist_returns(".global-loading-indicator",unit_test.settings.NORMAL_WAIT)

    if global_indicator_doesnt_exists:
        logger.debug("Global Loading Indicator button disappearing in 10 seconds")

    else:
        logger.warning("Global Loading Indicator button didn't disappear in 10 seconds")
        global_indicator_LONG_WAIT=unit_test.wait_element_doesnt_exist_returns(".global-loading-indicator",unit_test.settings.LONG_WAIT)

    unit_test.assertEqual(global_indicator_LONG_WAIT, True, "Global Loading Indicator button didn't disappear even after 40 seconds")

When time is surpassed, this throws an assertionException like so:

2014-09-30 12:27:34,315 - ERROR - Check_for_global_indicator_doesnt_exist() completed  - 40.236 sec
Traceback (most recent call last):
  File "C:\mypath\LogUtils\logwrap.py", line 71, in decorator
value = f(*args, **kwargs)
  File "C:\mypath\Shortcuts\Selectors.py", line 696, in Check_for_global_indicator_doesnt_exist
unit_test.assertEqual(global_indicator_LONG_WAIT, True, "Global Loading Indicator button didn't disappear even after 40 seconds")
  File "C:\Python27\lib\unittest\case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "C:\Python27\lib\unittest\case.py", line 508, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: Global Loading Indicator button didn't disappear even after 40 seconds

As I see it, there are two anomalies: 1) Why is nose considering these errors rather than failures? and 2) Why is the teardown() being prevented from firing??

Answer 1

After looking at many different examples of custom nose plugins, I found different plugins take a different number of arguments for their addSuccess, addError and addFailure functions.

My issue, as you may have guessed, was in the addFailure function. Nose wanted me to define the function with 5 args, but I only gave it space for 4.

Initial Code

def addSuccess(self, test, capt):
    do_stuff()

def addError(self, test, capt, tb_info):
    do_error_stuff()

def addFailure(self, test, capt, tb_info):
    do_failure_stuff()

Correct Code

...
def addFailure(self, test, err, capt, tb_info):
    do_failure_stuff()

It's unfortunate that the Nose docs on custom plugins is so sparse (In fact, the official nose docs only specify 3 arguments needed for addFailure, even though many of the "batteries included" plugins define it with 4). Without walking Nose's source code, I have no information in the docs about where these arguments are coming from. But at least it's solved!