Search code examples
c#.netunit-testingcode-coveragecoverlet

Coverlet test coverage: incorrectly use threshold


I have configured coverlet analysis of my .net code in this way:

  dotnet test \
    /p:CollectCoverage=true \
    /p:CoverletOutputFormat=cobertura \
    /p:Threshold=80 \
    /p:Exclude=[*]*.Program%2c[*]*.Controllers.*%2c[*]*.Models.*

with Threshold 80, however when I run the above script, I see that it fails even when the threshold bigger than 80, for example here:

Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 9 ms - ReactApp.Server.Tests.dll (net6.0)

Calculating coverage result...
  Generating report 'c:\..\ReactApp.Server.Tests\coverage.cobertura.xml'

+-----------------+------+--------+--------+
| Module          | Line | Branch | Method |
+-----------------+------+--------+--------+
| ReactApp.Server | 85%  | 50%    | 83.33% |
+-----------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 85%  | 50%    | 83.33% |
+---------+------+--------+--------+
| Average | 85%  | 50%    | 83.33% |
+---------+------+--------+--------+

not sure why? Any help would be appreciated.


Solution

  • Setting a single value requires all three measures (line, branch and method) to be above that value. As the branch coverage is less than 80, it fails.

    To split the coverage by measures, you need to format the value like this:

    /p:Threshold=80,50,80
    

    Otherwise, you need to add more tests to get your branch coverage to >= 80%.