I have just added Codeception to some of my projects but its performance is unacceptable for failing tests. Passing tests execute in 0.1s or less but failing tests take between 15 minutes and 1.5 hours each. When they finally fail, they dump hundreds of kilobytes to the console. There is a lot of mention of Selenium, which I don't understand because so far I'm only using unit tests, not acceptance tests or functional tests so I don't see why it should need Selenium. Why is it so slow? Is there something I can configure to make a failing test run as fast as a passing test? These are the tests I'm using:
<?php
class englishTest extends \Codeception\Test\Unit {
/**
* @var \UnitTester
*/
protected $tester;
protected function _before() {
include_once('country.model.php');
}
protected function _after() {
}
// tests
public function testCreateCountryModel() {
$country = new CountryModel;
$this->assertEquals('Afghanistan', $country->get_all_countries()[0]);
}
public function testGetCountriesEndingWithEndOfWord() {
$country = new CountryModel;
$this->assertEquals(['Costa Rica', 'Dominica', 'Jamaica', 'South Africa'], $country->get_countries_ending_with('ca'));
}
}
This is the output when the tests pass:
C:\server\Apache24\htdocs\localhost\public_html\english>codecept run
Codeception PHP Testing Framework v2.2.1
Powered by PHPUnit 4.8.26 by Sebastian Bergmann and contributors.
Acceptance Tests (0) ---------------------------------------
------------------------------------------------------------
Functional Tests (0) ---------------------------------------
------------------------------------------------------------
Unit Tests (2) ---------------------------------------------
+ englishTest: Create country model (0.1s)
+ englishTest: Get countries ending with end of word
------------------------------------------------------------
Time: 2.23 seconds, Memory: 11.00MB
OK (2 tests, 2 assertions)
I created a blank directory:
C:\server\Apache24\htdocs\localhost\public_html>md deception
I bootstrapped Codeception:
C:\server\Apache24\htdocs\localhost\public_html>codecept bootstrap deception
Initializing Codeception in C:\server\Apache24\htdocs\localhost\public_html\deception
File codeception.yml created <- global configuration
tests/unit created <- unit tests
tests/unit.suite.yml written <- unit tests suite configuration
tests/functional created <- functional tests
tests/functional.suite.yml written <- functional tests suite configuration
tests/acceptance created <- acceptance tests
tests/acceptance.suite.yml written <- acceptance tests suite configuration
---
tests/_bootstrap.php written <- global bootstrap file
Building initial Tester classes
Building Actor classes for suites: acceptance, functional, unit
-> AcceptanceTesterActions.php generated successfully. 0 methods added
\AcceptanceTester includes modules: PhpBrowser, \Helper\Acceptance
AcceptanceTester.php created.
-> FunctionalTesterActions.php generated successfully. 0 methods added
\FunctionalTester includes modules: \Helper\Functional
FunctionalTester.php created.
-> UnitTesterActions.php generated successfully. 0 methods added
\UnitTester includes modules: Asserts, \Helper\Unit
UnitTester.php created.
Bootstrap is done. Check out C:\server\Apache24\htdocs\localhost\public_html\deception/tests directory
I then changed to the directory and ran the following command in it:
C:\server\Apache24\htdocs\localhost\public_html\deception>codecept generate:test unit Deception
Test was created in C:\server\Apache24\htdocs\localhost\public_html\deception\tests\unit\DeceptionTest.php
I then added a single line to DeceptionTest.php ($this->assertTrue(false);
):
<?php
class DeceptionTest extends \Codeception\Test\Unit
{
/**
* @var \UnitTester
*/
protected $tester;
protected function _before()
{
}
protected function _after()
{
}
// tests
public function testMe()
{
$this->assertTrue(false);
}
}
Finally, I ran the following:
C:\server\Apache24\htdocs\localhost\public_html\deception>codecept run
Codeception PHP Testing Framework v2.2.1
Powered by PHPUnit 4.8.26 by Sebastian Bergmann and contributors.
Acceptance Tests (0) ---------------------------------------
------------------------------------------------------------
Functional Tests (0) ---------------------------------------
------------------------------------------------------------
Unit Tests (1) ---------------------------------------------
x DeceptionTest: Me
------------------------------------------------------------
Time: 3.36 minutes, Memory: 103.25MB
There was 1 failure:
---------
1) DeceptionTest: Me
Test tests\unit\DeceptionTest.php:testMe
Failed asserting that false is true.
#1 C:\server\Apache24\htdocs\localhost\public_html\deception\tests\unit\DeceptionTest.php:22
#2 DeceptionTest->testMe
#3 C:\usr\bin\codecept.phar:7
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
As you can see, it took 3.36 minutes and 103.25MB to work out that true !== false
.
I always use Xdebug with PHP. On a hunch I disabled it and suddenly, Codeception completes all tests, both passing and failing in only a few seconds. The culprit was xdebug.collect_params = 3
in php.ini. Setting it to any of 0, 1, or 2 fixes the problem.