I have a Jenkins job that runs nightly, it divides and allocates ~3000K to multiple machines and the allure report will be generated at the end collating all the allure-results folder from each machine. There are total 3100 test cases, and the number is fixed as always. But in some execution session, the generated allure report says there are total 2800, 2500, or even 1500 test cases being executed.
The reason seems to be because global-setup us run once in each of these machines, if the execution were to fail at the global-setup, allure does not save this result. I’m losing count on the legit total number of tests present.
Has anyone faced this issue and was able to resolve it with a customisation? Thanks in advance.
It appears this is the norm in allure reporting. Is there a way I could customize or contribute to making this change?
It turns out this was a bug from Allure. The team helped provide a hotfix. The issue will not be seen post 2.0.0.