I'm currently using Robot Framework to run our automated tests. My goal is to also re-run any failed tests and if any pass on the rerun, merge them into the final report. However, if any pass during the rerun part of my script, in the final generated report, report.html, it will only show those tests, and not the tests of the whole suite. Is there a way to fix this? I'm sure it's an issue with the script I'm running, but everything I've found online appears to be correct.
This is the script I'm currently running
Script:
python -m robot --outputdir tmp TestCases/Regression.robot || python -m robot --outputdir tmp --rerunfailed tmp/output.xml --output rerun.xml TestCases/Regression.robot || rebot --outputdir tmp --output output.xml --merge tmp/output.xml tmp/rerun.xml
If all of the tests that are re-ran fail a second time, it seems to merge the reports fine, and show both the old and new results on the generated report, report.html. It's only when a re-ran test passes that the report changes. Let me know if there's any additional info I can provide
I think it is because you merge your reports only if second robot fails. If all tests pass you will skip 3rd || expression.
Try to call this command instead:
python -m robot --outputdir tmp TestCases/Regression.robot || (python -m robot --outputdir tmp --rerunfailed tmp/output.xml --output rerun.xml TestCases/Regression.robot && rebot --outputdir tmp --output output.xml --merge tmp/output.xml tmp/rerun.xml)
It joins last two commands with && which means it will be executed both.
UPD: also you need to have 2 last commands in parenthesis because you need both of them if 1st robot fails and none of them if it succeeds.
UPD2: Actually this way || between 2 last commands will prevent merge if one of rerun tests fails.
I think you need to do merge separately like this:
python -m robot --outputdir tmp --report none --log none TestCases/Regression.robot || python -m robot --outputdir tmp --rerunfailed tmp/output.xml --output rerun.xml --report none --log none TestCases/Regression.robot
rebot --outputdir tmp --output output_final.xml --merge tmp/*.xml