cabal: how to stop the build on test failure?

Question

i created tests in HUnit (Tests.hs). i connected them to main: main = runTestTT tests. when i do runhaskell Tests i see

### Failure in: 0                          
T(1)
expected: 145
 but got: 45
Cases: 10  Tried: 10  Errors: 0  Failures: 1
Counts {cases = 10, tried = 10, errors = 0, failures = 1}

which is expected. in cabal file i did

test-suite xxx
  type: exitcode-stdio-1.0
  main-is: Tests.hs
  build-depends: base ==4.5.*, HUnit ==1.2.5.2, containers == 0.5.5.1

and when i do cabal test same test logs are written to a file - so i'm sure tests are executed and failing (as expected) but in console i see:

1 of 1 test suites (1 of 1 test cases) passed.

and the exit code is 0.

so my question is: why cabal claims tests passed and how to make it report errors correctly?

Answer 1

I just had to figure this out myself, this is what I finally got to work....

module Main where

import Data.Monoid
import Test.Framework
import Test.Framework.Providers.HUnit
import Test.HUnit

firstTest::Assertion --This one passes                                                                                                                                                                      
firstTest = do
  assertEqual "reward state root doesn't match" (1::Int) 1

secondTest::IO () --This one fails  (note, Assertion is just "IO()", so you can use either)                                                                                                                 
secondTest = do
  assertEqual "empty db didn't match" (1::Int) 2

main::IO ()
main =
  defaultMainWithOpts
  [
   testCase "ShortcutNodeData Insert" firstTest,
   testCase "FullNodeData Insert" secondTest
  ] mempty

In my .cabal file

Test-Suite test-program
  type:               exitcode-stdio-1.0
  main-is:            Main.hs
  hs-source-dirs:     test
  build-depends:      base
                , test-framework
                , test-framework-hunit
                , HUnit
                , containers

then run with cabal test

It is tricky, because cabal looks at the exitcode (see the type above), but the HUnit outputs its own messages.... So if you don't return the correct value, you can see output like "test failed" followed by "test passed". Obviously, the solution is to use the builtin defaultMainWithOpts, which does everything correctly.