Search code examples
azure-devopsazure-pipelinespester-5

Why does Pester in Classic Azure Pipeline always fail?

I have a classic build pipeline (not yaml, yet!) with three tasks

  • Install Pester -> works as expected
  • Run Pester tests -> always fails!
  • Publish test results -> works as expected

My script for "Run Pester tests"

Import-Module Pester -Force

$config = [PesterConfiguration]::Default
$config.TestResult.Enabled = $true
$config.TestResult.OutputFormat = "NUnitXml"
$config.TestResult.OutputPath = "$env:SYSTEM_DEFAULTWORKINGDIRECTORY\TEST-results.xml"
$config.Run.Path = ".\my\path*"
$config.Run.PassThru = $true

$results = Invoke-Pester -Configuration $config
if($results.Result -eq "Passed")
{
    Write-host "All tests passed"
    Write-Host "##vso[task.complete result=Succeeded;]DONE"
    exit 0
}

Sample output in Azure DevOps is

Tests completed in 11.43s
Tests Passed: 67, Failed: 0, Skipped: 0 NotRun: 0
All tests passed
Finishing: Run Pester Unit Tests

However, the task always fails

enter image description here

I know from robocopy that Powershell Tasks in Azure will check $LASTEXITCODE and that you are able to overwrite that via Write-Host "##vso[task.complete result=Succeeded;]DONE" and exit 0. Why is this not working here?

What else have I tried?

  • Activate SilentlyContinue -> results in Warning
  • Activate Continue on Error -> results in Warning
  • Activate Ignore $LASTEXITCODE -> results in Failure
  • Activate Fail on Standard Error -> results in Failure
Solution

  • Oh boy this was a tough one: The comment by Shayki Abramczyk enabled me to track it down. Is it possible to grant him the bounty somehow?

    The cause was a unit test, that was changing the Azure DevOps task output via Write-Host "##vso[task.complete result=Failed;]DONE". Since my Powershell scripts are supposed to work in Azure DevOps, they should do things like this. The test was named nicely for Pester it "Should print out error if build requirement not found"

    My learning here

    1. Make your issues smaller. Reducing this to a single test, showed me directly that one of my tests was the cause even though they all passed.
    2. You cannot test the following line in Azure DevOps with Pester, because if will always make your tests fail Write-Host "##vso[task.complete result=Failed;]DONE"
    3. A test executing that line, will show "Passed" which can be correct, but is annoying and hard to find.
    4. You cannot overwrite that by the following line afterwards Write-Host "##vso[task.complete result=Succeeded;]DONE"