Is it possible to run a "final stage" in Azure Pipelines without knowing how many stages there are in total?

Question

Is there a way to let a stage run as the very last stage (not including the post-job/ Report build status)?

The thing is: I have an each-loop with an unkown amount of elements, every element gets their own stage.

so we can't just say "dependendOn" on the final stage, because every previous stage has an unique name given at runtime.

The stages should sequence as followed:

1. Prepare

2 - n) Build / Compile / Test with specific docker container

last) Remove all containers

Currently this works fine, with the small issue, that the last stage will run after the last of the n-Stages is over (done with succeededOrFailed condition) The problem is, when the last of the n-Stages is faster than all the others.

The Remove container stage is separated, because we ran into issues, when all the n-stages tried removing their containers at the same time, which docker can not handle.

The pipeline runs on yaml files

Answer 1

Is it possible to run a “final stage” in Azure Pipelines without knowing how many stages there are in total?

The answer is yes.

The main idea of the solution is:

In the last stage (Remove all containers), add a Inline powershell task before Remove all containers. This task will call the REST API Definitions - Get to monitor whether all the stages in the current release pipeline have inprocess and queue states. If so, wait for 30 seconds, and then loop again until all other stages in the current release pipeline have no inprocess and queue states. Then next Remove all containers task will be executed.

The scripts:

For example, I add Inline powershell task in my Dev stage with following scripts:

$connectionToken="$(Your PAT here)"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
Start-sleep -Seconds 20 # In case this stage run as first stage.
$success = $false
$count = 0

do{
    try{

        $stageurl2 = "https://vsrm.dev.azure.com/{org name}/{project name}/_apis/release/deployments?definitionId={release definition ID}&deploymentStatus=inProgress&api-version=6.0"
        $stageinfo2 = (Invoke-RestMethod -Uri $stageurl2 -Method Get -UseDefaultCredential -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}) 

        $stageInProgressNum= $stageinfo2.count

        $stageurl3 = "https://vsrm.dev.azure.com/{org name}/{project name}/_apis/release/deployments?definitionId={release definition ID}&deploymentStatus=notDeployed&api-version=6.0"
        $stageinfo3 = Invoke-RestMethod -Method Get -ContentType application/json -Uri $stageurl3 -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}

        $stageInQueuedNum = $stageinfo3.count

        Write-Host "stage In Progress Num is = $stageInProgressNum"

        Write-Host "stage In Queued Num is = $stageInQueuedNum"

        if($stageInProgressNum -ne 1) {     #The value set to 1 is because the ststus of curret stage is in inProgress when this powershell scripts is executed.      
            Write-output "There is/are $stageInProgressNum Deployment In Progress, Waiting for it to finish!"
            Write-output "Next attempt in 30 seconds"
            Start-sleep -Seconds 30         
      } else {           
            Write-Host "No Current Deployment in Progress"
            if($stageInQueuedNum -ne 0) {
            write-output "There is/are $stageInQueuedNum Queued deployments"
            Write-output "Next attempt in 30 seconds"
            Start-sleep -Seconds 30
            }
            else{
            write-output "No Queued deployments, starting release"
            $success = $true     
            }     
      }
    }
    catch{
        Write-output "catch - Next attempt in 30 seconds"
        write-output "1"
        Start-sleep -Seconds 30
      # Put the start-sleep in the catch statemtnt so we
      # don't sleep if the condition is true and waste time
    }
   
    $count++
   
}until($count -eq 2000 -or $success)
if(-not($success)){exit}

The test result:

Dev will continue to check until all other stages are complete:

Note: You need add the Inline powershell task in the latest stage, I added it in the middle of the stages to clearly show that the inline powershell task will monitor the entire release before the other stages are completed.

Update:

I tried to convert this to Build Pipelines instead of Release Pipelines. However I couldn't find any API reference to the status of Build-Stages. The doc for stages is empty learn.microsoft.com/en-us/rest/api/azure/devops/build/… And postman doesn't want to filter on the definitions in build area

If you want use this for build pipeline, there is no documented REST API to get the stage status at this moment. But we could press F12 in the browser then select Network to capture the request to get the stage result. You can capture the result from the response body. But different stage results are represented by different numbers i.e 0->Not start, 1->inprocess, 2->complete, etc:

https://dev.azure.com/<YourOrgName>/LeoTest/_build/results?buildId=6522&__rt=fps&__ver=2

Now, we can use the same idea to determine the value of state to solve your needs. I will not share the code repeatedly here, if you have any further questions, please let me know.