In our build there are certain scenarios that fail for reasons which are out of our control or take too long to debug properly. Things such asynchronous javascript etc.
Anyway the point is sometimes they work sometimes they don't, so I was thinking it would be nice to add a tag to a scenario such as @rerun_on_failure or @retry which would retry the scenarion X number of times before failing the build.
I understand this is not an ideal solution, but the test is still valuable and we would like to keep it without having the false negatives
The actual test that fails clicks on a link and expects a tracking event to be sent to a server for analytics (via javascript). Sometimes the selenium web-driver loads the next page too fast and the event does not have time to be sent.
Thanks
I've been considering writing something like what you're describing, but I found this:
If you're tired of having to re-kick builds in your CI server because of non deterministic failures, this post is for you.
In a nutshell: he makes a new rake task called cucumber:rerun that uses rerun.txt to retry failed tests. It should be pretty easy to add some looping in there to retry at most 3x (for example).