Search code examples
pythonunit-testingpytestpython-hypothesis

Using hypothesis and py.test to test compound strategies in python, must I test them one at a time?


I have 3 files, module.py which contains an example function that tests whether an input is numeric. I have a file called test_mymodule_long.py that successfully tests and passes several types of inputs using py.test and hypothesis. I'm using Python 3.6, but this shouldn't matter (if you're not, just remove the type hinting). For this particular function, it's not helpful to me to have these split up; if hypothesis finds an edge case, I just want to know the offending input and the exception raised. Thus, I'd like to write this test as indicated in test_mymodule_short.py, but this doesn't work.

My question is this: Is there a way to write strategies in hypothesis more efficiently than in test_mymodule_long.py?

Here is the code in each file:

'''
mymodule.py
'''

from typing import Union

Number = Union[int, float]


def is_numeric(x: Number):
    try:
        y = float(x)
        return True
    except:
        return False

# -------------------------------------------

'''
test_mymodule_long.py [This code works]
(using pytest to discover tests automatically)
'''
from hypothesis import given, example
import hypothesis.strategies as st
import mymodule

@given(st.integers())
def test_is_numeric_integer(num):
    result = stats.is_numeric(num)
    assert isinstance(result, bool)


@given(st.floats())
def test_is_numeric_float(num):
    result = stats.is_numeric(num)
    assert isinstance(result, bool)


@given(st.text())
def test_is_numeric_text(num):
    result = stats.is_numeric(num)
    assert isinstance(result, bool)


@given(st.lists(st.floats()))
def test_is_numeric_list(num):
    result = stats.is_numeric(num)
    assert isinstance(result, bool)

# -------------------------------------------

'''
test_mymodule_short.py [This code fails!]
(using pytest to discover tests automatically)
'''
from hypothesis import given, example
import hypothesis.strategies as st
import mymodule

@given(st.integers())
@given(st.floats())
@given(st.text())
@given(st.lists(st.floats()))
def test_is_numeric_list(num):
    result = mymodule.is_numeric(num)
    assert isinstance(result, bool)

Note that I don't think the error message really matters here, it's more of a this-isn't-how-you-do-it situation and I'm seeking advice on whether there is a compact way to test one function with multiple hypothesis strategies. In addition, I know that a proper is_numeric function would be programmed differently (assuming you really even needed such a function). I also know that the tests indicated here would not be sufficiently complete to know whether it worked. These are just examples to make my question clear.


Solution

  • There's no way to use multiple strategies for a single test, but you can combine multiple strategies into a single one and use that in your tests - the one_of strategy. So you could write something like the following:

    from hypothesis import given, example
    import hypothesis.strategies as st
    import mymodule
    
    @given(st.one_of(
        st.integers(), 
        st.floats(),
        st.text(),
        st.lists(st.floats()),
    ))
    def test_is_numeric_list(num):
        result = mymodule.is_numeric(num)
        assert isinstance(result, bool)