Search code examples
haskellhaskell-stack

How to run an individual test with Stack and Haskell Test.Framework?


I'm cloning the haskoin/haskoin-core repository and making one change to the stack.yaml by adding at the end:

docker:
    enable: true

To run all the tests for haskoin-core I'm using

stack test haskoin-core:test-haskoin-core

What I want to do is run just one test. If this were HSpec (which it is not) I'd be running something like:

stack test --test-arguments -m "Network.Haskoin.Network.Units"

Now what I could do is modify the file haskcoin-core/test/Main.hs and comment out all the tests I don't want to run. But ya know - there should be a simpler way to run it with just command line parameters. (Mutating the file system goes agains the whole functional grain of Haskell).

I'd also be open to running it with stack ghci somehow.

My question is: How to run an individual test with Stack and Haskell Test.Framework?


Solution

  • Thanks to @sjakobi for this answer.

    The process is - list the available test commands:

    stack test --test-arguments "--help" haskoin-core:test-haskoin-core
    

    This gives the following result:

    haskoin-core-0.4.2: test (suite: test-haskoin-core, args: --help)
    
    Usage: test-haskoin-core [OPTIONS]
                       --help                                       show this help message
      -j NUMBER        --threads=NUMBER                             number of threads to use to run tests
                       --test-seed=NUMBER|random                    default seed for test random number generator
      -a NUMBER        --maximum-generated-tests=NUMBER             how many automated tests something like QuickCheck should try, by default
                       --maximum-unsuitable-generated-tests=NUMBER  how many unsuitable candidate tests something like QuickCheck should endure before giving up, by default
      -s NUMBER        --maximum-test-size=NUMBER                   to what size something like QuickCheck should test the properties, by default
      -d NUMBER        --maximum-test-depth=NUMBER                  to what depth something like SmallCheck should test the properties, by default
      -o NUMBER        --timeout=NUMBER                             how many seconds a test should be run for before giving up, by default
                       --no-timeout                                 specifies that tests should be run without a timeout, by default
      -l               --list-tests                                 list available tests but don't run any; useful to guide subsequent --select-tests
      -t TEST-PATTERN  --select-tests=TEST-PATTERN                  only tests that match at least one glob pattern given by an instance of this argument will be run
                       --jxml=FILE                                  write a JUnit XML summary of the output to FILE
                       --jxml-nested                                use nested testsuites to represent groups in JUnit XML (not standards compliant)
                       --plain                                      do not use any ANSI terminal features to display the test run
                       --color                                      use ANSI terminal features to display the test run
                       --hide-successes                             hide sucessful tests, and only show failures
    
    
    Test suite failure for package haskoin-core-0.4.2
        test-haskoin-core:  exited with: ExitFailure 1
    Logs printed to console
    

    From that we can build a command to list the tests:

    stack test --test-arguments "--list-tests" haskoin-core:test-haskoin-core
    

    From there we can use this command to glob for a particular test

    stack test --test-arguments=--select-tests=Bloom*Filter haskoin-core:test-haskoin-core
    

    Note the * in place of the space, there seems to be some discussion about how to handle spaces in this scenario.

    Now this selects the tests we want to run:

    haskoin-core-0.4.2: test (suite: test-haskoin-core, args: --select-tests=Bloom*Filter)
    
    Binary encoding and decoding of bloom types:
      BloomFilter: [OK, passed 100 tests]
    Bloom Filters:
      Bloom Filter Vector 1: [OK]
      Bloom Filter Vector 2: [OK]
      Bloom Filter Vector 3: [OK]
    
             Properties  Test Cases  Total      
     Passed  1           3           4          
     Failed  0           0           0          
     Total   1           3           4