Search code examples
testingautomated-testse2e-testingtestcafeweb-testing

Testcafe tests ran by automation servers fail randomly


This is inconsistent behavior, I am getting random results with both Jenkins and GitLab-runner. When running the failing tests on the same build machine, everything goes well. (e.g. the tests that failed when ran by the automation server, pass when ran manually - using test.only)

Tried mostly headless with Firefox and Chrome but the same randomness happens with full UI as well.

First off, I thought there was a resource problem on the build machine because of concurrent tasks, but I ruled this off by scheduling a nightly build. Moreover, I even reduced the speed to 0.8.

Has anyone else encountered this behavior? Any hint will be greatly appreciated.


Solution

  • Try to enable the quarantine mode and try to skip javascript errors.

    Without a specific error message is difficult to pin-point the cause