I'm running pytest through a Python script, and I want it to know if the pytest has failed. The intuitive way for me would be to catch any stderr on pytest's exit, but I just can't seem to find a way to do it. Any tips?
My tests look like this:
def test_2():
object = Object()
object.func()
assert object.property == expected_value
The tests are called like this:
pytest.main([path_to_test, "-s"])
I want to know whether even a single assert has failed, and do something about it, maybe like so:
err = io.StringIO()
with contextlib.redirect_stderr(err):
pytest.main([path_to_test, "-s"])
So basically I want pytest to throw anything in stderr when it's done if any test has failed.
A solution I've found is to replace asserts with a function that immediately throws an error, but then not all tests are ran.
Edit: OK, I just had another idea, which looks like this:
def assert_test(test_case):
if not test_case:
sys.stderr.write("failure")
assert test_case
I guess I found something that works for me, but I still find it weird that there's no built-in or prettier solution...
Edit edit: OK, so that solution messes up the logging. I've resorted to monkey patching the assert... IDK if that's a great idea.
class AssertionError():
def __init__(self, *args, **kwargs):
sys.stderr.write("failure")
raise builtins.AssertionError
Now the calling script is aware if there was a failure.
pytest.main()
returns the exit code as a result, so if you're looking for a pass/fail on the entire run you can check the return value, just like you would in a shell script. It is very coarsely-grained, however, it is just the one pass/fail value for the entire test run.
According to the docs, the following exit codes are defined:
- Exit code 0: All tests were collected and passed successfully
- Exit code 1: Tests were collected and run but some of the tests failed
- Exit code 2: Test execution was interrupted by the user
- Exit code 3: Internal error happened while executing tests
- Exit code 4: pytest command line usage error
- Exit code 5: No tests were collected