I have a bunch of rspec tests and roughly 30 of those are headless tests through capybara/poltergeist. Every time i run the tests locally, all 100% of the tests pass.
I recently set up a teamcity continuous integration server and the test results have been extremely inconsistent.
I ran the tests 10 times, without doing any changes between the tests. Only 1 of the rounds passed 100%. The others had 1-2 failures (most of them weren't the same), all relating to the headless browser testing. Here is an example of one of the failures:
ActionView::Template::Error: Couldn't find Spree::Address with id=1072978592
Stack trace:
./app/models/spree/order_decorator.rb:50:in `initialize_default_address'
./app/models/spree/order_decorator.rb:42:in `initialize_shipment'
./app/views/layouts/core.html.erb:23:in `_8cb98e121af585621c1d08e3ec1f6022'
./app/views/layouts/default.html.erb:14:in `_588b3208edc213a939dffd2ad73f4f26'
This failure is odd because i stubbed the function that looks for an address with an ID and returns a factorygirl model. I do not have this problem locally.
Here is another error from one of the test runs:
Capybara::ExpectationNotMet: expected to find link "10" but there were no matches
Stack trace:
./spec/features/simply_ship_spec.rb:102:in `block (2 levels) in <top (required)>'
Again, when i run this locally, this error doesn't happen, and when i switch to selenium and watch the test, the link is clearly there.
Both of these errors (not all, but a couple examples) happened individually, on different runs, without doing any updates to the project. Does anybody have any ideas why this is happening?
rspec-rails 2.14.0 capybara 2.1 poltergeist 1.3.0 rails 3.2.14 ruby 1.9.3 p448
Turns out there is a bug in poltergeist that is causing the unreliable clicks. you can follow the bug discussion here: