I test my Symfony2 project with PHPUnit and behat/mink. This morning I added more behat tests to my project and all of a sudden random tests started failing. It seems like I hit a limit (memory? execution time? something else?) but I can't find anything.
Travis-ci does not fail on this http://travis-ci.org/#!/tvlooy/ReadingCorner/builds/383016
My local machine does fail (Lenovo R60, 2GB RAM, 2x 1.8Ghz CPU). I run Fedora 16 on x86, 32bit. PHP 5.3.8 (cli), with Xdebug v2.1.2, PHPUnit 3.6.2.
(edit: I just updated to PHPUnit 3.6.4 and removed Xdebug but the problem remains)
I run my Behat tests with:
$ app/console behat -e=test
=> random tests fail
When I run the tests like this:
$ for feature in 'find src/Ctors/ReadingCornerBundle/Features/ -name *.feature'; do app/console behat -e=test $feature; done
=> all tests pass
I already raised my memory_limit in php.ini to 768M but it didn't help.
I have 7 feature files with a total of 57 scenario's. Not much, nothing exotic. The files are in https://github.com/tvlooy/ReadingCorner/tree/master/src/Ctors/ReadingCornerBundle/Features
I need the tests to work on my local machine in one command because I generate code coverage from it with PHPUnit (wrapped behat features in PHPUnit).
Any clues?
@jakub_zalas blogged about a fix. See http://www.zalas.eu/fixing-failing-behat-scenarios-in-large-suites Setting 1002: 'SET SESSION wait_timeout=30;' fixed it for me too.