Search code examples
pythonunit-testinghypothesis-test

Should I instantiate Hypothesis strategies that are used by multiple tests?


The built-in hypothesis strategies are provided via functions (e.g. rather than an actual strategy, integers is a function that creates a strategy). That suggests to me that strategy objects have internal state.

@given(integers())
def test_foo(n):
    assert n > 0    
@given(integers())
def test_bar(n):
    assert n < 100

In the two bogus tests above, each test gets a different strategy object (from different invocations of integers. If I then create my own strategy like this:

positive_integers = integers().filter(lambda x: x > 0)

... then use it for the same tests:

@given(positive_integers)
def test_foo(n):
    assert n > 0    
@given(positive_integers)
def test_bar(n):
    assert n < 100

They share the same strategy object. This sounds to me like it might be wrong, but it's the way the examples in the docs do it in some cases (see definition of NodeStrategy and NodeSet). Should I avoid this by wrapping the strategy composition in a function like:

 positive_integers = lambda: integers().filter(lambda x: x > 0)
 #...
 @given(positive_integers())

Solution

  • I took a look at the source code, and it looks like you should be fine to share the same strategy object across tests. It seems like you call a function so you can pass in different parameters for the strategy.

    I think that means you could do either this:

    @given(integers(min_value=0))
    def test_foo(n):
        assert n > 0    
    @given(integers(min_value=0))
    def test_bar(n):
        assert n < 100
    

    or this:

    positive_integers = integers(min_value=0)
    
    @given(positive_integers)
    def test_foo(n):
        assert n > 0    
    @given(positive_integers)
    def test_bar(n):
        assert n < 100
    

    I couldn't see any evidence of state beyond the range boundaries. In fact, the BoundedIntStrategy seems to get the search data passed in as a parameter:

    def do_draw(self, data):
        return d.integer_range(data, self.start, self.end)
    

    However, I've only played a little bit with Hypothesis, so I could certainly be wrong.