How to fail early (e.g. set a timeout) on slow tests in python3 (preferably using nose)

Question

The following code reproduces my situation:

from nose.tools import timed
from time import sleep


class Test():
    @timed(1)
    def test_slow_function(self):
        duration = 5
        sleep(duration)
        pass

Running the test (e.g. nosetests test.py:Test -s), I would expected a failure result after just 1 second. To my surprise, it does not fail until the test finishes (in this case after 5 seconds). Even if any result after 1 second will already be a failure. I get:

...
    raise TimeExpired("Time limit (%s) exceeded" % limit)
nose.tools.nontrivial.TimeExpired: Time limit (1) exceeded

----------------------------------------------------------------------
Ran 1 test in 5.006s

FAILED (failures=1)

I want to avoid the possibility that the test suite never ends (e.g. there is an infinite loop under certain circumstances). Which would be a good approach?

Answer 1

@timed decorator cant stop execution of decorated function. All it does is just compares the real execution time with the expected one, and raises a Fail if it's exceeded.

Basically, to monitor some process, and stop it in some case (if too long, in your example), you need another process in parallel, that will actually do the monitoring. The easy and a bit hacky way you can achieve this is by using nose's parallel test execution, like this:

$ nosetests test.py:Test -s --processes 2 --process-timeout 1
E
======================================================================
ERROR: timesout_nose.Test.test_slow_function
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/test.py", line 9, in test_slow_function
    sleep(duration)
  File "/venv/lib/python3.6/site-packages/nose/plugins/multiprocess.py", line 276, in signalhandler
    raise TimedOutException()
nose.plugins.multiprocess.TimedOutException: 'test.Test.test_slow_function'

----------------------------------------------------------------------
Ran 1 test in 1.230s

FAILED (errors=1)

You can read more here: http://nose.readthedocs.io/en/latest/plugins/multiprocess.html

However you wont be able to setup an easy time limit, like you did with the decorator. But you can still catch infinite loops with that.