How to assert both UserWarning and SystemExit in pytest

Question

Assert UserWarning and SystemExit in pytest

In my application I have a function that when provided with wrong argument values will raise a UserWarnings from warnings module and then raises SystemExit from sys module.

Code is something like:

def compare_tags(.....):

    requested_tags = user_requested_tags # as list
    all_tags = tags_calculated_from_input_file  # as list 

    non_matching_key = [x for x in requested_tags if x not in all_tags]

    # if user requested non existing tag then raise warning and then exit 
    if len(non_matching_key) > 0:

        # generate warning
        warnings.warn("The requested '%s' keys from '%s' is not present in the input file. Please makes sure the input file has the metadata of interest or remove the non matching keys." %(non_matching_key, given_tags))

        # raise system exit
        sys.exit(0)

writing a pytest for above function

I want to test this UserWarning and SystemExit in pytest at once. I can check for SystemExit in the pytest as.

with pytest.raises(SystemExit):
    compare_tags(....)

but this will also dispaly a warning message (which is not an error).

If I want to check for warnings:

pytest.warns(UserWarning, 
    compare_tags(...)

This will generate a SystemExit error because this called function will trigger system exit.

How can I put both the warnings and SystemExit check in the same pytest?

Answer 1

pytest.warns and pytest.raises are the usual context managers and can be declared in a single with statement when separated with a comma (see compound statements):

with pytest.warns(UserWarning), pytest.raises(SystemExit):
    compare_tags(...)

which is effectively the same as writing

with pytest.warns(UserWarning):
    with pytest.raises(SystemExit):
        compare_tags(...)

Notice that the order matters - when you put both context managers in the reverse order:

with pytest.raises(SystemExit), pytest.warns(UserWarning):
    ...

this is the same as writing

with pytest.raises(SystemExit):
    with pytest.warns(UserWarning):
        ...

The problem here is that pytest.raises will capture all raised errors and then check for what's captured. This includes what pytest.warns raises. This means that

with pytest.raises(SystemExit), pytest.warns(UserWarning):
    sys.exit(0)

will pass because the error raised in pytest.warns will be swallowed in pytest.raises, while

with pytest.warns(UserWarning), pytest.raises(SystemExit):
    sys.exit(0)

will fail as expected.