Search code examples
pythonpytestpython-hypothesis

Why does my simple, finite hypothesis test never stop?

I am running a test suite with hypothesis-4.24.6 and pytest-5.0.0. My test has a finite set of possible inputs, but hypothesis never finishes testing.

I have reduced it to the following minimal example, which I run as pytest test.py

from hypothesis import given
import hypothesis.strategies as st


@given(x=st.just(0)
         | st.just(1),
       y=st.just(0)
         | st.just(1)
         | st.just(2))
def test_x_y(x, y):
    assert True

I would expect it to try all six combinations here and then succeed. Or possibly a small multiple of that to check for flakiness. Instead it runs indefinitely, (after about 15 mins of testing I kill it.)

If I interrupt the test, back traces seem to show it just continuously generating new examples.

What have I done wrong here?

Solution

  • This bug was fixed in Hypothesis 4.26.2, or at least we thought so; it's actually fixed in 4.26.3 though: https://hypothesis.readthedocs.io/en/latest/changes.html#v4-26-3