Search code examples
haskellquickcheck

How to run a specific number of tests in QuickCheck?


I am trying to learn Haskell and specifically QuickCheck. While Haskell has a lot of information online I am struggling to create some random test with QuickCheck.

For example, I have the following script:

import Test.QuickCheck

whatAge :: Int -> Int -> Int -> Int -> Bool

whatAge age1 age2 age3 age4
  | age1 + age2 + age3 + age4 == 5 = True
  | otherwise = False

main = do
    verboseCheck  whatAge

When I run it shows:

*** Failed! Falsifiable (after 1 test): 
0
0
0
0

Fairly enough it showed a test on which the function was false.

What I would like to do though is to:

  1. Generate 200 random tests even on failure (a.k.a even when the output of the whatAge function is false)
  2. Be able to put a range on my function parameters, for example:

       x1 range from 1 to 30
    
       x2 range from 1 to 40
    
       x3 range from 1 to 50
    
       x4 range from 1 to 60
    
  3. Be able to generate non-repeating tests

From my understanding nr 3 is not really possible with QuickCheck, for that I will have to use smallCheck but I am not sure about point 1 and 2.


Solution

  • For simple properties of your input, you can make a newtype with an appropriate Arbitrary instance that captures them. So:

    {-# LANGUAGE AllowAmbiguousTypes #-}
    {-# LANGUAGE DataKinds #-}
    {-# LANGUAGE GeneralizedNewtypeDeriving #-}
    {-# LANGUAGE KindSignatures #-}
    {-# LANGUAGE ScopedTypeVariables #-}
    {-# LANGUAGE TypeApplications #-}
    
    import Data.Proxy
    import GHC.TypeLits
    import Test.QuickCheck
    
    newtype Range (m :: Nat) (n :: Nat) a = Range { getVal :: a }
        deriving (Eq, Ord, Read, Show, Num, Real, Enum, Integral)
    
    numVal :: forall n a. (KnownNat n, Num a) => a
    numVal = fromInteger (natVal @n Proxy)
    
    instance (KnownNat m, KnownNat n, Arbitrary a, Integral a) => Arbitrary (Range m n a) where
        arbitrary = fromInteger <$> choose (numVal @m, numVal @n)
        shrink hi = go (numVal @m) where
            go lo | lo == hi = [] | otherwise = lo : go ((lo+hi+1)`div`2) -- overflow? what's that? lmao
    
    whatAge :: Range 1 30 Int -> Range 1 40 Int -> Range 1 50 Int -> Range 1 60 Int -> Bool
    whatAge (Range age1) (Range age2) (Range age3) (Range age4)
        = age1 + age2 + age3 + age4 == 5
    

    In ghci:

    > verboseCheck whatAge
    Failed:  
    Range {getVal = 17}
    Range {getVal = 29}
    Range {getVal = 3}
    Range {getVal = 16}
    
    Failed:                                  
    Range {getVal = 1}
    Range {getVal = 29}
    Range {getVal = 3}
    Range {getVal = 16}
    
    Failed:                                               
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 3}
    Range {getVal = 16}
    
    Failed:                                                
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 16}
    
    Failed:                                                
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 1}
    
    *** Failed! Falsifiable (after 1 test and 4 shrinks):  
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 1}
    Range {getVal = 1}
    

    For more complicated properties, where it's not clear to how to directly create a random value that satisfies the property, you may use QuickCheck's (==>) operator. For example, for range checks as above:

    > verboseCheck (\x -> (1 <= x && x <= 30) ==> x*2 < 60)
    Skipped (precondition false):
    0
    
    Passed:                
    1
    
    *** Failed! Falsifiable (after 33 tests):                  
    30
    

    To make exactly 200 tests, you could call quickCheckWith to make one test, 200 times; or you could directly generate test results by calling your property on arbitrary manually.