Search code examples
azure-devopsazure-devops-rest-api

Azure DevOps REST API: Match Manual Test Case steps to Test Run results?


I would like to create a simple report for manual test results of test cases using REST API:

  • Test case step description / expected result
  • Result: pass / fail
  • Comments, attachments

I use the following query

http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results?api-version=5.1&detailstoinclude=Iterations

(The latest supported API version on our server is 5.1)

The part of the response is below.

My problem is that the actionResults are not in the order of test case steps. Looking at the comments, I can see that step 6 actionResult goes after step 2, then step 8, then step 4 etc. This is further complicated by the fact that we use shared steps.

Is there a way I could match the test case steps to the corresponding results from the iteration details?

{
"count": 1,
"value": [
    {
        "id": 100000,

        ...

        "iterationDetails": [
            {
                "id": 1,
                "outcome": "Failed",
                "errorMessage": "",
                "durationInMs": 171574,
                "actionResults": [
                    {
                        "actionPath": "00000002",
                        "iterationId": 1,
                        "url": "http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results/100000/Iterations/1/ActionResults/00000002",
                        "outcome": "Failed",
                        "errorMessage": "step 1 comment",
                    },
                    {
                        "actionPath": "00000003",
                        "iterationId": 1,
                        "url": "http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results/100000/Iterations/1/ActionResults/00000003",
                        "outcome": "Failed",
                        "errorMessage": "step 2 comment",
                    },
                    {
                        "actionPath": "00000004",
                        "iterationId": 1,
                        "url": "http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results/100000/Iterations/1/ActionResults/00000004",
                        "outcome": "Failed",
                        "errorMessage": "step 6 comment",
                    },
                    {
                        "actionPath": "00000005",
                        "iterationId": 1,
                        "url": "http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results/100000/Iterations/1/ActionResults/00000005",
                        "outcome": "Failed",
                        "errorMessage": "step 8 comment",
                    },
                    {
                        "actionPath": "00000006",
                        "iterationId": 1,
                        "url": "http://myserver.is.mycompany.net:8080/tfs/MyCollection/LMTest/_apis/test/Runs/192557/Results/100000/Iterations/1/ActionResults/00000006",
                        "outcome": "Failed",
                        "errorMessage": "step 4 comment",
                    },
                    
                    ....
        }
    }
]

}


Solution

  • According to Results - List,

    actionPath is the path identifier for test step in test case workitem.

    It increments in the order in which the steps are added to the test case, not in the order in which the steps actually appear in the test case.

    For example, you added the first step "StepA" to the test case. At this time, the actionPath of this step is 00000002. Then you inserted a new step "StepB" before "StepA". At this time, the actionPath of "StepB" is 00000003. The actionPath of "StepA" remains unchanged, which is still 00000002.

    enter image description here

    In the API response body, the object in actionResults are sorted in ascending order by actionPath.

    enter image description here

    The actual execution order is to run StepB first, then run StepA.

    enter image description here