Search code examples
python-hypothesis

Collecting sample statistics in the test

I am wondering if there's a way to report statistics of examples in the test using Hypothesis. In other PBTs framework, such as PropEr, provide a way (in this case collect/2) to generate a report of examples in the test. For example, the following code in Erlang using PropEr:

prop_dupes() ->
    ?FORALL(KV, list({key(), val()}),
      begin
        M = maps:from_list(KV),
        [_ = maps:get(K, M) || {K, _V} <- KV],
        collect(
            {dupes, to_range(5, length(KV) - length(lists:ukeysort(1,KV)))},
            true
        )
      end).

It should generate the following output:

OK: Passed 100 test(s).

82.00% {dupes,{0,5}}
12.00% {dupes,{5,10}}
 5.00% {dupes,{10,15}}
 1.00% {dupes,{15,20}}
===>

Is this something that hypothesis.event can achieve?

Solution

  • Yes, that's exactly what hypothesis.event() is designed for - see https://hypothesis.readthedocs.io/en/latest/details.html#hypothesis.event for details.