Search code examples
haskelltestingtastyprogress-reports

Progress reporting in Haskell Tasty tests

I have a bunch of tests in my tasty test suite. Currently tasty runs all the tests and then print out the results all at once, e.g.

Tests
  DeBruijnIndices
    substUnderLambdaClosed:       OK
    liftUnderLambdaOpenType:      OK
  Normalisation
    normaliseLambda:              OK
    normaliseAppLambdaClosedBody: OK
    normaliseAppLambdaOpenBody:   OK

However, sometimes I create an infinite loop by accident and one of the tests fails to terminate. In this case tasty never prints anything, even though it's running the tests in a multi-threaded environment. This makes it impossible to diagnose which test is the problem.

Is there someway to make tasty print out the intermediate progress on the tests? The first entry of the CHANGELOG for v1.5 suggests there might be but I have no idea how to hook into it, and there doesn't seem to be any examples or documentation provided.

Solution

  • You have to use cabal test tests --test-show-detail=streaming

    Apparently in the future (likely 3.12) a new detail level will be the defailt, namely direct which is like streaming but it does not create a log file but does have colored output:

    https://cabal.readthedocs.io/en/latest/setup-commands.html#cmdoption-runhaskell-Setup.hs-test-test-show-details

    Alternatively you could use cabal run tests, which runs the tests as a normal executable.