We have a Story that is linked to the Cucumber Test and we execute this Test in Test Execution. The feature file from Test Execution looks like this:
@ABC-164
@ABC-163
Feature: Homepage test
@ABC-162
Scenario: Homepage test
Feature: Go to homepage
Scenario: Homepage
Given I open the url "https://example.com/"
Then I expect that the url is "https://google.com"
The first one is always passed and the second one failed. We are generating a JSON report from wdio cucumber json reporter, which you can see below:
[
{
"keyword": "Feature",
"type": "feature",
"description": "",
"line": 3,
"name": "Homepage test",
"uri": "Can not be determined",
"tags": [
{
"type": "Tag",
"location": { "line": 1, "column": 1 },
"name": "@ABC-164"
},
{
"type": "Tag",
"location": { "line": 2, "column": 1 },
"name": "@ABC-163"
}
],
"elements": [
{
"keyword": "Scenario",
"type": "scenario",
"description": "\t\tFeature: Go to homepage",
"name": "Homepage test",
"tags": [
{ "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
{ "name": "@ABC-163", "location": { "line": 2, "column": 1 } },
{ "name": "@ABC-162", "location": { "line": 6, "column": 2 } }
],
"id": "homepage-test;homepage-test",
"steps": [
{
"arguments": [],
"keyword": "Before",
"name": "Hook",
"result": { "status": "passed", "duration": 2000000 },
"line": "",
"match": { "location": "can not be determined with webdriver.io" }
},
{
"arguments": [],
"keyword": "Before",
"name": "Hook",
"result": { "status": "passed", "duration": 0 },
"line": "",
"match": { "location": "can not be determined with webdriver.io" }
}
]
},
{
"keyword": "Scenario",
"type": "scenario",
"description": "",
"name": "Homepage",
"tags": [
{ "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
{ "name": "@ABC-163", "location": { "line": 2, "column": 1 } }
],
"id": "homepage-test;homepage",
"steps": [
{
"arguments": [],
"keyword": "Before",
"name": "Hook",
"result": { "status": "passed", "duration": 0 },
"line": "",
"match": { "location": "can not be determined with webdriver.io" }
},
{
"arguments": [],
"keyword": "Given",
"name": "I open the url \"https://example.com/\"",
"result": { "status": "passed", "duration": 587000000 },
"line": 12,
"match": { "location": "can not be determined with webdriver.io" }
},
{
"arguments": [],
"keyword": "Then",
"name": "I expect that the url is \"https://google.com\"",
"result": {
"status": "failed",
"duration": 11000000,
"error_message": "Error: expect(received).toEqual(expected)"
},
"line": 13,
"match": { "location": "can not be determined with webdriver.io" }
},
{
"arguments": [],
"keyword": "After",
"name": "Hook",
"result": { "status": "passed", "duration": 0 },
"line": "",
"match": { "location": "can not be determined with webdriver.io" }
}
]
}
],
"id": "homepage-test",
"metadata": {
"browser": { "name": "chrome", "version": "88.0.4324.150" },
"device": "Device name not known",
"platform": { "name": "osx", "version": "Version not known" }
}
}
]
Xray only reads the status from Before Hooks from the first object and totally ignore the rest of the statuses.
The problem occurs when we use Jenkins Plugin and also with direct sending via curl like that: curl -k -H "Content-Type: application/json" -X POST -u login:pass --data @result.json https://jira/rest/...
Any help would be appreciated.
For those who will have the same problem. We managed to find the solution. Thank you very much for the answers!
It looks like we had an old structure of the feature file in Jira (back then when we didn't have stories and preconditions). When we start to use it properly: linked Stories to Tests and add Pre-condition then Xray created doubled parts of scenarios. So we deleted the redundant parts from Cucumber Tests, create proper Pre-condition, and now the feature file is created properly like that:
@ABC-169
@ABC-131
Feature: Main feature title
Background:
#@ABC-166
Given I open the url "https://example.com/"
Then I expect that the url is "https://google.com"
@ABC-132 @ABC-168 @ABC-141
Scenario: Main scenario
Given Lorem ipsum
Then Et sit amet