Search code examples
catch-unit-test

Containing the combinatorial explosion of test cases using Catch


Let us say I have a simple class which takes 6 boolean arguments in its constructor and performs some computation based on the state of those arguments.

If I want to use Catch to adequately test all of the cases, then I would need 64 separate unit tests.

Now, let's say at some point in the future, I add a 7th boolean argument. Now the number of test cases that I have to write has doubled to 128.

Is there a way that I can craft my unit test in such as a way as to automatically "generate" all 2^n test cases from a single test specification?


Solution

  • You can use the generators part of Catch to automatically walk through all the different combinations of bools that your function takes:

    TEST_CASE("where is my sandwich", "[hunger][food]")
    {
      bool wantLettuce = GENERATE(Values(false, true));
      bool wantTomato = GENERATE(Values(false, true));
      bool wantBacon = GENERATE(Values(false, true));
      bool wantCheese = GENERATE(Values(false, true));
      bool wantEgg = GENERATE(Values(false, true));
    
      CHECK(sandwichAssembler(wantLettuce, wantTomato, wantBacon, wantCheese, wantEgg));
     }
    

    IIRC this will cause Catch to run 2^5 times trying all the combinations.

    ...but I'm assuming that all you want to do is exercise all the combinations. If you actually want to verify the output meets some expectation in each case, you'll need to do more.

    (I've not tested this - recalling from memory while waiting for my sandwich in a cafe in Barcelona)