Pitest - Test Coverage Report Data

Question

I understand that when outputting the test coverage report from Pitest, you are presented with the following structure:

<block classname='classname' method='methodName' number='1'>
   <tests>
      <test name='testName1(file1)'/>
      <test name='testName2(file2)'/>
   </tests>
</block>

However, I am unsure of the meaning of the 'number' attribute. What is the significance of it? Thanks

Answer 1

The number is the number of the block within the method.

Blocks are chunks of code that execute together. To calculate coverage, pitest creates a probe in each block. If the probe is executed, pitest then knows that all instructions within that block have been executed, unless an exception is thrown.

Most coverage systems place the probe at the end of the block, so in event of an exception the coverage will be underreported. Pitest places them at the beginning of each block. This causes overreporting, but this is desirable for pitest as the coverage data is used to target tests against mutants. It is better to run a test and it not to detect a mutant than not run one which would.

The precise definition of a block has changed between pitest releases. Originally, it roughly corresponded to branches in the code.

int foo(boolean b) {
  System.out.println("hello"); // block 1
  int i = 2; // block 1
  if (b) { // block 1
    System.out.println("boo"); // block 2
    i = i + 42; // block 2
  }
  System.out.println("bye"); // block 3
  return i; // block 3
} 

So the code above would have three blocks as shown in the comments. Later releases have made the blocks smaller so tests can be targeted more accurately when exceptions occur.