Search code examples
testingazure-devopscontinuous-integrationcontinuous-deploymentazure-pipelines-yaml

How to identify and push the last build that passed all manual testing to UAT/Prod

Goal

I have a similar goal/question to this one, where on a regular basis (e.g. weekly), we need to able to isolate and promote the last build that successfully passed all manual testing (not just automated).

Background/Problem

The project I am on has multiple Azure deployments (e.g. Dev, Test, UAT, Prod) which are mapped to Environments in DevOps. All work is managed and tracked to User Stories/Tasks against multiple repos.

We have YAML multi-stage pipelines to build (once) then deploy to each environment in sequence. The pipeline automatically builds and deploys to "Dev" after each accepted Pull Requests (PR). Deployment to "Test" is done via a manual trigger and remaining Environments (e.g. UAT, Prod ) have approval checks in place to delay deployment until the relevant QA's have completed all "manual" tests within each environment.

We are using Test Plans for internal QA/testing of builds deployed within "Test" environment. Then pass User Stories to the Product Owners (PO's) who will mark the story as Complete when it meets their approval.

Over the course of each sprints, lots of new functionality/changes get built & deployed to "Dev" & "Test". There will be times where everything in the current build passes all manual tests, but sadly, that's not always the case, and so more work is needed.

When the time comes that a new release to UAT is needed - we will not always be able to push the latest build currently in "Test", as it may not have passed all manual tests. Whereas, there should (hopefully) have been a time earlier in the sprint (with less functionality) where all manual tests were passed.

Does anyone have any suggestions/guidance on how to make the best use of Azure DevOps so that we can quickly identify the most recent build that previously "passed" all manual tests?

(Alternatively, very open to suggestions of ways to achieve the same, simpler!)

Many thanks in advance
(+please let me know if any of the above needs clarifying)

Solution

  • We can link work items to builds with these steps.

    If you want to auto link builds to the work items, you can refer the following steps:

    1. Click the Repositories and the Branches Menu, choose the branch you are going to use to build from and click the options icon. Select Branch policies.enter image description here
    2. Check the option "Check for linked work items". enter image description here
    3. As you are using the YAML multi-stage pipelines, open the pipeline, choose More actions, and then choose Settings.

    enter image description here

    1. Enable Automatically link new work in this build.enter image description here

    2. Once enabled, Integrated in build links are generated for all work items linked to the selected pull request with each run.

    enter image description here

    For more details, you can refer Configure pipelines to support work tracking.

    Update

    I found this blog: How to retrieve all work items associated with a release pipeline using Azure DevOps API, it mentions:

    Although you can use some work item links types as part of your query parameters, by design it is not possible to query by commits, builds or releases. The reason is those links are treated internally in the tool as a relation object instead of the conventional link objects such as Parent and Child link types.

    The solution is to use the Azure DevOps REST API.

    So, we can use this REST API Builds - Get Build Work Items Refs From Commits to get the linked work items.

    Here is my test PowerShell script.

    Param (
    [string]$organization = "YourOrganization",
    [string]$project = "YourProject",
    [string[]]$buildIds = @("YourBuildId1", "YourBuildId2"),  # Add more build IDs as needed
    [string]$token = "YourPAT"
)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "",$token)))

# Loop through each build ID
foreach ($buildId in $buildIds) {
    $workItemsUrl = "https://dev.azure.com/$organization/$project/_apis/build/builds/$buildId/workitems?api-version=7.1-preview.2"
    $workItemsResponse = Invoke-RestMethod -Uri $workItemsUrl -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method Get
    
    # Check if there are any work items linked to the build
    if ($workItemsResponse.count -eq 0) {
        Write-Output "The Build ID $buildId does not link to any workitem."
        Write-Output "--------------------------------------------------------"
        continue
    }

    # Initialize a flag to check if all work items are resolved or closed
    $allResolvedOrClosed = $true

    # Loop through each work item
    foreach ($workItemRef in $workItemsResponse.value) {
        $workItemUrl = $workItemRef.url + "?api-version=7.1-preview.3"
        $workItemResponse = Invoke-RestMethod -Uri $workItemUrl -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method Get

        Write-Output ("Build ID: " + $buildId + ", Work Item ID: " + $workItemResponse.id + ", Title: " + $workItemResponse.fields.'System.Title' + ", Status: " + $workItemResponse.fields.'System.State')

        # Check if the work item is not resolved or closed
        if ($workItemResponse.fields.'System.State' -ne "Resolved" -and $workItemResponse.fields.'System.State' -ne "Closed") {
            $allResolvedOrClosed = $false
        }
    }

    # Check if all work items are resolved or closed
    if ($allResolvedOrClosed) {
        Write-Output "The Build ID $buildId passed."
        Write-Output "--------------------------------------------------------"
    }
    else{
        Write-Output "The Build ID $buildId didn't pass yet."
        Write-Output "--------------------------------------------------------"
    }
}

    My test result:

    enter image description here