Why does QuickCheck give up?

Question

I am using QuickCheck to test the following program:

{-# LANGUAGE TemplateHaskell #-}

import Test.QuickCheck
import Test.QuickCheck.All

elementAt :: (Integral b) => [a] -> b -> a
elementAt [x] _ = x
elementAt (x:xs) 1 = x
elementAt (x:xs) b = elementAt xs (b - 1)

prop_elementAt xs b = length xs > 0 && b >= 0 && b < length xs ==> elementAt xs (b + 1) == xs !! b

main = $(quickCheckAll)

Although the response varies, I constantly receive the message

*** Gave up! Passed only x tests.

Is this something I should be concerned about? Or does the nature of the test input dictate how long QuickCheck will run for?

Answer 1

The way ==> works is first quickcheck will generate random values for xs and b, then check if the predicate length xs > 0 && b >= 0 && b < length xs is satisfied only then it will check for the satisfiability of the property.

As there is a limit for how many test cases it will generate it might happen that lots of the time the above predicate is not satisfied. So quickcheck gives up before generating enough valid testcases (satisfying the predicate).

You should instead declare Arbitrary instance to a newtype to generate only testcases satisfying those predicates.

{-# LANGUAGE TemplateHaskell #-}

import Test.QuickCheck
import Test.QuickCheck.All

elementAt :: (Integral b) => [a] -> b -> a
elementAt [x] _ = x
elementAt (x:xs) 1 = x
elementAt (x:xs) b = elementAt xs (b - 1)

prop_elementAt (Foo xs b) = elementAt xs (b + 1) == xs !! b

data Foo a b = Foo [a] b deriving (Show)

instance (Integral b, Arbitrary a, Arbitrary b) => Arbitrary (Foo a b) where
  arbitrary = do
    as <- listOf1 arbitrary           -- length xs > 0
    b <- choose (0,length as - 1)     -- b >= 0 and b < length xs
    return (Foo as $ fromIntegral b)

main = $(quickCheckAll)