How to Run Multiple Test Files with Haskell Stack Project

Question

I want to setup an existing Haskell project with Stack. The existing project uses multiple files under a test directory; these separate test files by default, Stack (or cabal?) appears to utilize a single test/Spec.hs for testing. How can I continue to use multiple files with this project?

NOTE: I'm learning Haskell, and this project approaches my learning from a "kata" approach. So tests are isolate to focus on one aspect of the language at a time.

Answer 1

Here is a setup for a directory structure like this

> tree                                                                                       
.
├── example.cabal
├── app
│   └── Main.hs
├── ChangeLog.md
├── LICENSE
├── Setup.hs
├── src
│   ├── A
│   │   └── C.hs
│   ├── A.hs
│   └── B.hs
├── stack.yaml
└── tst
    ├── integration
    │   └── Spec.hs
    └── unit
        ├── A
        │   └── CSpec.hs
        ├── ASpec.hs
        ├── BSpec.hs
        └── Spec.hs

you want to have integration tests that are separate from the usual unit tests and several sub-modules that correspond to each module in your src-folder

first of all you need to add the test suites to your

example.cabal file

name:                example
...
-- copyright:
-- category:
build-type:          Simple
extra-source-files:  ChangeLog.md
cabal-version:       >=1.10

executable testmain
  main-is:       Main.hs
  hs-source-dirs: app
  build-depends: base
               , example

library
  exposed-modules:     A.C,A,B
  -- other-modules:
  -- other-extensions:
  build-depends:       base >=4.9 && <4.10
  hs-source-dirs:      src
  default-language:    Haskell2010

test-suite unit-tests
  type:          exitcode-stdio-1.0
  main-is:       Spec.hs
  hs-source-dirs: tst/unit
  build-depends: base
               , example
               , hspec
               , hspec-discover
               , ...

test-suite integration-tests
  type:          exitcode-stdio-1.0
  main-is:       Spec.hs
  hs-source-dirs: tst/integration
  build-depends: base
               , example
               , hspec
               , ...

put the following in your tst/unit/Spec.hs it is from hspec-discover and it discovers (hence the name) all modules of the form ...Spec.hs and executes the spec function from each of those modules.

tst/unit/Spec.hs

{-# OPTIONS_GHC -F -pgmF hspec-discover #-}

just this single line

Other test files

then add your unit tests in your ASpec.hs, and others in BSpec.hs,CSpec.hs and your Spec.hs in the tst/integration folder

module ASpec where

import Test.Hspec
import A

spec :: Spec
spec = do
  describe "Prelude.head" $ do
    it "returns the first element of a list" $ do
      head [23 ..] `shouldBe` (23 :: Int)

    it "returns the first element of an *arbitrary* list" $
      property $ \x xs -> head (x:xs) == (x :: Int)

    it "throws an exception if used with an empty list" $ do
      evaluate (head []) `shouldThrow` anyException

you can then compile and run your tests with

$> stack test
# now all your tests are executed
$> stack test :unit-tests
# now only the unit tests run
$> stack test :integration-tests
# now only the integration tests run

Sources

You can find all the examples at https://hspec.github.io, if you want to know more about hspec-style testing I guess it would be best to start there. For the stack - go to https://haskellstack.org - there is some information about testing/benchmarking there - I mean about running tests and benchmarks.

For different testing style in haskell see HUnit, QuickCheck, Smallcheck, doctests (If I forgot one, my dearest apologies - those are the ones that I use regularly as well).