Defining a suite of tests for a class

Question

This question picks up where Haskell QuickCheck best practices (especially when testing type classes) left off.

I have a class and a bunch of implementations of that class. Something like this:

import Test.QuickCheck
import Control.Applicative
import Test.Framework
import Test.Framework.Providers.QuickCheck2

class C c where
  f :: c -> Int

data A = A Int deriving Show

instance C A where
  f (A a) = 2*a

data B = B Int deriving Show

instance C B where
  f (B b) = 2*b

All of my implementations should satisfy a certain property. For example:

prop_f_is_even :: C c => c -> Property
prop_f_is_even x = property $ even (f x)

I want to test that property for each of the implementations. I can do something like this. (I'm using Test.Framework.)

instance Arbitrary A where
  arbitrary = A <$> arbitrary

instance Arbitrary B where
  arbitrary = B <$> arbitrary

test :: Test
test = testGroup "Whole buncha tests"
  [
    testProperty "prop_f_is_even - A" (prop_f_is_even :: A -> Property),
    testProperty "prop_f_is_even - B" (prop_f_is_even :: B -> Property)
    -- continue on for all combinations of properties and implementations
  ]

But in my case, I have dozens of properties to test, and a dozen or so classes, so that approach is error-prone, and a hassle. (A common mistake I make is to cut-and paste tests, but forget to change the type name, so I end up testing A twice for that property, without testing B.)

I have a solution, which I'll post below in case anyone else finds it helpful.

Answer 1

This is my solution.

cProperties :: C t => String -> [(String, t -> Property)]
cProperties s = 
  [
    ("prop_f_is_even: " ++ s, prop_f_is_even)
    -- plus any other tests that instances of C should satisfy
  ]

makeTests :: (Arbitrary t, Show t) => [(String, t -> Property)] -> [Test]
makeTests ts = map (\(s,t) -> testProperty s t) ts

aProperties :: [(String, A -> Property)]
aProperties = cProperties "A"

bProperties :: [(String, B -> Property)]
bProperties = cProperties "B"

easierTest :: Test
easierTest = 
  testGroup "tests" (makeTests aProperties ++ makeTests bProperties)

With this approach, if I want to add another property that all instances of C should satisfy, I just add it to cProperties. And if I create another instance of C, call it D, then I define dProperties similarly to aProperties and bProperties, and then update easierTest.

---

EDIT: One disadvantage of this approach is that all tests in cProperties have to have the type signature t -> Property. I myself have not found this to be a hindrance because in cases where I apply this technique, I have already -- for unrelated reasons -- defined a type that encompasses all of the data for a test.

Another disadvantage is that, in ghci, I can no longer type, for example:

quickCheck prop_f_is_even

Now I have to type something like this:

quickCheck (prop_f_is_even :: A -> Property)