Search code examples
unit-testinghaskellcabal

Code coverage in a Haskell project that only builds an executable

I have a cabal project that builds an executable.

When I run cabal test --enable-coverage I get the following error:

All 9 tests passed (0.98s)
Test suite giter-test: PASS
Test suite logged to:
/home/refaelsh/repos/giter/dist-newstyle/build/x86_64-linux/ghc-9.2.7/giter-0.1.0/test/giter-0.1.0-giter-test.log
Error: .cabal-wrapped: Test coverage is only supported for packages with a
library component.

Error: cabal: Tests failed for giter-0.1.0.
  1. Can somebody please explain what is going on?
  2. Is that error a normal thing to happen when my cabal project has only an executable and no libraries?
  3. Should I just make a library of those part I wanna Unit Test and make my exe dependent on that library?
    1. If yes, is that the common practice in idiomatic Haskell?
    2. If yes, it feels like a code smell to make changes to source code and/or project structure to accommodate Unit Tests.
Solution

  • I think the underlying technical problem is covered in this issue. Cabal assumes that you don't want a coverage report on the unit test code itself, just the code being tested, so it invokes hpc in such a way that unit test code is excluded and only the source code actually being tested is included in coverage reporting. This is easy to do if the code is part of a separately compiled library: the library is covered, and any modules that are part of the unit testing are not covered.

    However, if you have no library stanza, and only executable and test-suite stanzas that use a shared set of modules, there's no obvious way to distinguish the test-suite modules that are part of the unit testing and should not be covered from the modules that are part of the code being tested that should be covered.

    More generally, Cabal has poor support for testing an executable stanza. As per the discussion in this issue, the only way of actually testing an executable is to use a hack -- either specify executable and test-suite stanzas that share the same hs-source-dirs which ends up double-compiling everything, or else move everything important into a library and leave your executable as a stub. You are probably using the former approach, rather than the latter, but both kind of suck. The developers seem to have decided that the "move everything into a library" is the recommended approach, even if not all of the documentation and/or tutorial material makes this clear.

    Note that the situation for Stack is similar -- the default templates typically have a stub application containing only a main function with everything else moved into a library.

    I guess I would recommend moving your entire application (not just the parts you want to test) into a library, and writing a stub main function for your executable like:

    module Main where

import qualified Lib

main :: IO ()
main = Lib.main

    with both the test-suite and executable depending on the library.