Search code examples
c#.netpowershellazure-devopsopencover

Exact same OpenCover script completes successfully when run locally, but fails when run via Azure DevOps


Until a few days ago, I have been using OpenCover to run my unit tests and output code coverage without fail. I have a powershell script that I can run locally and also use in an Azure DevOps build and have not had any issues. But now, OpenCover is not working ONLY on Azure DevOps.

I have my computer configured as a Build Agent in my Azure DevOps instance running as a service as the same user that I am logged in as when I successfully run OpenCover locally.

$info = New-Object System.Diagnostics.ProcessStartInfo
$info.FileName = "C:\Users\devuser\.nuget\packages\opencover\4.6.519\tools\OpenCover.Console.exe"
$info.RedirectStandardError = $true
$info.RedirectStandardOutput = $true
$info.UseShellExecute = $false
$info.Arguments = "`
                -target:""C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"" `
                -targetargs:""C:\agent\_work\3\s\UnitTests\bin\Debug\UnitTests.dll"" `
                -output:""C:\agent\_work\3\s\opencover.xml"" `
                -filter:""+[*]*.Service.* +[*]*.Model.* +[*]*.Repository.*"" `
                -excludebyfile:*""*\*Designer.cs"" `
                -mergebyhash `
                -skipautoprops `
                -returntargetcode `
                -register:user"

$exec = New-Object System.Diagnostics.Process
$exec.StartInfo = $info
$exec.Start() | Out-Null
$exec.StandardOutput.ReadToEnd() | Out-File -FilePath "C:\agent\_work\3\s\log.txt"
$exec.StandardError.ReadToEnd() | Out-File -FilePath "C:\agent\_work\3\s\log_errors.txt"

When running the above script locally, I get this expected result:

Executing: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
Microsoft (R) Test Execution Command Line Tool Version 16.2.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
  û GetBookByBookID [636ms]
  û SelectBooksByUser [132ms]
  û SelectAllBooks [64ms]
  û GetBookIDByCoinID [70ms]
  û SelectCoinsByBook [79ms]
  û PingS3 [922ms]
  û GetCoinByCoinID [140ms]
  û SelectCoinsBySearch [66ms]
  û AddAndRemoveCoin [383ms]
  û SelectAllCoins [170ms]
  û SelectAllCoinsByBookByUser [135ms]
  û SelectOwnedCoinsByBook [128ms]
  û SelectAllMintmarks [63ms]
  û SelectAllYears [75ms]
  û SelectAllDenominations [63ms]
  û SelectAllLinkUserCoin [189ms]
  û SelectUsers [125ms]
  û GetUserNameByUserID [62ms]
  û GetUserIDByUserName [69ms]
  û SelectUsersByOwned [189ms]
  û SelectAllLinkUserBook [67ms]

Test Run Successful.
Total tests: 21
     Passed: 21
 Total time: 6.0269 Seconds
Committing...
Visited Classes 11 of 11 (100)
Visited Methods 57 of 57 (100)
Visited Points 336 of 336 (100)
Visited Branches 105 of 113 (92.92)

==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 11 of 11 (100)
Alternative Visited Methods 62 of 62 (100)

However running the exact same script through Azure DevOps, I get this:

Executing: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
Microsoft (R) Test Execution Command Line Tool Version 16.2.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
  √ GetBookByBookID [471ms]
  √ SelectBooksByUser [136ms]
  √ SelectAllBooks [62ms]
  √ GetBookIDByCoinID [62ms]
  √ SelectCoinsByBook [66ms]
  √ PingS3 [417ms]
  √ GetCoinByCoinID [133ms]
  √ SelectCoinsBySearch [66ms]
  √ AddAndRemoveCoin [413ms]
  √ SelectAllCoins [134ms]
  √ SelectAllCoinsByBookByUser [142ms]
  √ SelectOwnedCoinsByBook [129ms]
  √ SelectAllMintmarks [63ms]
  √ SelectAllYears [64ms]
  √ SelectAllDenominations [61ms]
  √ SelectAllLinkUserCoin [148ms]
  √ SelectUsers [134ms]
  √ GetUserNameByUserID [63ms]
  √ GetUserIDByUserName [61ms]
  √ SelectUsersByOwned [225ms]
  √ SelectAllLinkUserBook [64ms]

Test Run Successful.
Total tests: 21
     Passed: 21
 Total time: 4.8670 Seconds
Committing...
No results, this could be for a number of reasons. The most common reasons are:
    1) missing PDBs for the assemblies that match the filter please review the
    output file and refer to the Usage guide (Usage.rtf) about filters.
    2) the profiler may not be registered correctly, please refer to the Usage
    guide and the -register switch.

I have googled the two suggested reasons but am already using

-register:user

And obviously the pdb's are there because the build config is Debug and it runs as expected locally, both under the same user.

Any suggestions? I am at a loss at what could be different/missing on Azure DevOps and why this is happening after months of using this script successfully...


Solution

  • The solution is to use:

    -register:Path32
    

    rather than:

    -register:user
    

    Reference: https://github.com/OpenCover/opencover/issues/915#issuecomment-526284026