phpphpunitcode-coveragephp-code-coverage
Why Do PhpUnit dataProviders Lower Coverage Scores?
I have been using PhpUnit with good results, testing my code with assertions only. Recently I decided to try PhpUnit with coverage report analysis but I noticed that tests that use dataProvider methods tend to decrease my code coverage scores. I wonder what I might be doing wrong or if this is a consequence of dataProvider testing techniques? I am using PhpUnit 6 with Php 7.
I have included a source class, Foo, with three test classes below that test it. FooTest uses regular test methods, no dataProviders. BarTest uses dataProvider methods with @codeCoverageIgnore annotations and BazTest uses dataProvider methods with no annotations.
You can see how the code coverage score is lower with BazTest.
Foo.php
namespace phpunittestproject\src;
/**
* Foo
*
* This is a simple class to be used as a source file
* in unit test experiments. It gets and sets a name
* string and date object.
*
*/
class Foo
{
/**
* Name
*
* String characters other than numeric.
*
* @var string
*/
private $name = null;
/**
* Date
*
* Date no older than 2000.
*
* @var \DateTime
*/
private $date = null;
/**
* Constructor
*
* Sets instance vars.
*
*/
public function __construct()
{
$this->name = '';
$this->date = new \DateTime('now');
}
/**
* Get Date
*
* @return DateTime
*/
public function getDate()
{
return $this->date;
}
/**
* Get Name
*
* @return string
*/
public function getName()
{
return $this->name;
}
/**
* Set Date
*
* This method accepts a DateTime object that corresponds
* to a date no earlier than 2000.
*
* @param \DateTime $value Date after 2000.
* @return boolean Result of operation.
*/
public function setDate(\DateTime $value)
{
if($value < new \DateTime('2000-01-01 00:00:00')){
return false;
} else {
$this->date = $value;
return true;
}
}
/**
* Set Name
*
* This method accepts a string that does not contain numeric
* characters.
*
* @param string $value String without numeric characters.
* @return boolean Result of operation.
*/
public function setName(string $value)
{
if(preg_match('/\\d/', $value)){
return false;
} else {
$this->name = $value;
return true;
}
}
}
FooTest.php:
declare(strict_types = 1);
namespace phpunittestproject\test;
use \phpunittestproject\src\Foo;
/**
*
* Foo Test
*
* This test class does not use dataProvider methods. All
* test assertions are being done in test metods.
*
*/
class FooTest extends \PHPUnit\Framework\TestCase
{
/**
*
*
*/
public function testGetDate()
{
$foo = new Foo();
// Good date:
$date = new \DateTime('2011-01-01 11:11:11');
$foo->setDate($date);
$this->assertEquals($date, $foo->getDate());
// Bad date:
$date = new \DateTime('1990-01-01 11:11:11');
$foo->setDate($date);
$this->assertNotEquals($date, $foo->getDate());
}
/**
*
*
*/
public function testGetName()
{
$foo = new Foo();
// Good name:
$foo->setName('A Good Name');
$this->assertEquals('A Good Name', $foo->getName());
// Bad name:
$foo->setName('Bad Name 666');
$this->assertNotEquals('Bad Name 666', $foo->getName());
}
/**
* Test setDate()
*
* This test method tests the date property when it is
* set with good and bad data using method setDate().
*
*/
public function testSetDate()
{
$foo = new Foo();
// Good date:
$date = new \DateTime('2011-01-01 11:11:11');
$foo->setDate($date);
$this->assertAttributeEquals($date, 'date', $foo);
// Bad date:
$date = new \DateTime('1990-01-01 11:11:11');
$foo->setDate($date);
$this->assertAttributeNotEquals($date, 'date', $foo);
}
/**
*
* Test setName()
*
* This test method tests the name property when it is
* set with good and bad data using method setName().
*
*
*/
public function testSetName()
{
$foo = new Foo();
// Good name:
$foo->setName('Good Name');
$this->assertAttributeEquals('Good Name', 'name', $foo);
// Bad name:
$foo->setName('Bad Name 666');
$this->assertAttributeNotEquals('', 'name', $foo);
}
}
BarTest.php:
declare(strict_types = 1);
namespace phpunittestproject\test;
use \phpunittestproject\src\Foo;
/**
*
* Bar Test
*
* This test class utilizes dataProvider methods to feed
* test methods. The dataProvider methods are annotated
* with codeCoverageIgnore.
*
*/
class BarTest extends \PHPUnit\Framework\TestCase
{
/**
*
* @codeCoverageIgnore
*/
public function providerTestSetDateWithInvalidData()
{
return array(
array(new \DateTime('1990-01-01 11:11:11')),
);
}
/**
*
* @codeCoverageIgnore
*/
public function providerTestSetDateWithValidData()
{
return array(
array(new \DateTime('2011-01-01 11:11:11')),
);
}
/**
*
* @codeCoverageIgnore
*/
public function providerTestSetNameWithInvalidData()
{
return array(
array('Bad Name 666'),
);
}
/**
*
* @codeCoverageIgnore
*/
public function providerTestSetNameWithValidData()
{
return array(
array('Good Name'),
);
}
/**
*
*
*/
public function testGetDate()
{
$foo = new Foo();
$date = new \DateTime('2001-01-01 11:11:11');
$foo->setDate($date);
$this->assertEquals($date, $foo->getDate());
}
/**
*
*
*/
public function testGetName()
{
$foo = new Foo();
$foo->setName('A Good Name');
$this->assertEquals('A Good Name', $foo->getName());
}
/**
*
* @dataProvider providerTestSetDateWithInvalidData
*
*
*/
public function testSetDateWithInvalidData($value)
{
$foo = new Foo();
$foo->setDate($value);
$this->assertAttributeNotEquals($value, 'date', $foo);
}
/**
*
* @dataProvider providerTestSetDateWithValidData
*
*
*/
public function testSetDateWithValidData($value)
{
$foo = new Foo();
$foo->setDate($value);
$this->assertAttributeEquals($value, 'date', $foo);
}
/**
*
* @dataProvider providerTestSetNameWithInvalidData
*
*
*/
public function testSetNameWithInvalidData($value)
{
$foo = new Foo();
$foo->setName($value);
$this->assertAttributeNotEquals($value, 'name', $foo);
}
/**
*
* @dataProvider providerTestSetNameWithValidData
*
*
*/
public function testSetNameWithValidData($value)
{
$foo = new Foo();
$foo->setName($value);
$this->assertAttributeEquals($value, 'name', $foo);
}
}
BazTest.php:
declare(strict_types = 1);
namespace phpunittestproject\test;
use \phpunittestproject\src\Foo;
/**
*
* Baz Test
*
* This test class utilizes dataProvider methods to feed
* test methods. The dataProvider methods are not annotated
* with codeCoverageIgnore.
*
*/
class BazTest extends \PHPUnit\Framework\TestCase
{
/**
*
*
*/
public function providerTestSetDateWithInvalidData()
{
return array(
array(new \DateTime('1990-01-01 11:11:11')),
);
}
/**
*
*
*/
public function providerTestSetDateWithValidData()
{
return array(
array(new \DateTime('2011-01-01 11:11:11')),
);
}
/**
*
*
*/
public function providerTestSetNameWithInvalidData()
{
return array(
array('Bad Name 666'),
);
}
/**
*
*
*/
public function providerTestSetNameWithValidData()
{
return array(
array('Good Name'),
);
}
/**
*
*
*/
public function testGetDate()
{
$foo = new Foo();
$date = new \DateTime('2001-01-01 11:11:11');
$foo->setDate($date);
$this->assertEquals($date, $foo->getDate());
}
/**
*
*
*/
public function testGetName()
{
$foo = new Foo();
$foo->setName('A Good Name');
$this->assertEquals('A Good Name', $foo->getName());
}
/**
*
* @dataProvider providerTestSetDateWithInvalidData
*
*
*/
public function testSetDateWithInvalidData($value)
{
$foo = new Foo();
$foo->setDate($value);
$this->assertAttributeNotEquals($value, 'date', $foo);
}
/**
*
* @dataProvider providerTestSetDateWithValidData
*
*
*/
public function testSetDateWithValidData($value)
{
$foo = new Foo();
$foo->setDate($value);
$this->assertAttributeEquals($value, 'date', $foo);
}
/**
*
* @dataProvider providerTestSetNameWithInvalidData
*
*
*/
public function testSetNameWithInvalidData($value)
{
$foo = new Foo();
$foo->setName($value);
$this->assertAttributeNotEquals($value, 'name', $foo);
}
/**
*
* @dataProvider providerTestSetNameWithValidData
*
*
*/
public function testSetNameWithValidData($value)
{
$foo = new Foo();
$foo->setName($value);
$this->assertAttributeEquals($value, 'name', $foo);
}
}
phpunit.xml:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit colors="true" bootstrap="./vendor/autoload.php">
<testsuites>
<testsuite name="DataProviderTestSuite">
<file>phpunittestproject/test/FooTest.php</file>
<file>phpunittestproject/test/BarTest.php</file>
<file>phpunittestproject/test/BazTest.php</file>
</testsuite>
</testsuites>
<filter>
<whitelist>
<file>phpunittestproject/test/FooTest.php</file>
<file>phpunittestproject/test/BarTest.php</file>
<file>phpunittestproject/test/BazTest.php</file>
</whitelist>
</filter>
</phpunit>
Solution
Thanks @Christopher for the solution! A misconfigured phpunit.xml file was causing PhpUnit to test the tests. Editing the whitelist elements fixed the low code coverage score. The dataProvider methods do not need to be annotated with @codeCoverageIgnore!
<?xml version="1.0" encoding="UTF-8"?>
<phpunit colors="true" bootstrap="./vendor/autoload.php">
<testsuites>
<testsuite name="DataProviderTestSuite">
<!-- Test files go here: -->
<file>phpunittestproject/test/FooTest.php</file>
<file>phpunittestproject/test/BarTest.php</file>
<file>phpunittestproject/test/BazTest.php</file>
</testsuite>
</testsuites>
<filter>
<whitelist>
<!-- Source files to be tested go here: -->
<file>phpunittestproject/src/Foo.php</file>
</whitelist>
</filter>
</phpunit>
- Build a string by repeating a character N times
- Validate an array as "empty" if it has no elements or if all elements have no length
- phpunit runs test twice - gets two answers. Why?
- How can I make Laravel return a custom error for a JSON REST API
- regex for checking URL's wildcard for both subdomain and top level domain?
- Use Simple HTML Dom to parse HTML and generate an array of href values and plain text
- Generate an array of href strings from all <a> tags in an HTML document using Simple HTML DOM Parser
- Making POST application/json request with file_get_contents
- Woocommerce add to cart button redirect to checkout
- PHP form send email to multiple recipients
- How to loop over json-encoded data?
- Fill array with numbers adding up to a nominated total without exceeding item maximum
- Iterate over all last-day-of-the-month dates in a year
- Generate an associative 2d array with incremented column values using a flat array as first level keys
- Debug a single PHP file with $_GET parameters in NetBeans/Eclipse/PHPstorm (AJAX API)
- Put Values in Multidimensional Array keys
- Laravel queued jobs processed immediately even with a delay
- Error when calling function it self, VCS say="undefined function"
- How to truncate number 3 decimals
- PHP - How to get OAuth 2.0 Token
- Consume API endpoint from Laravel + Inertia App
- Generate real US street addresses for Selenium RC script
- Hour difference between 2 DateTime objects in Decimal Format
- Php update database if checkbox is not checked
- Group data from lines of a log file by one column and create subarrays from another column
- Use values of a flat array as the keys in every row of a 2d array
- Redis keys are not expiring - Laravel, Predis
- WooCommerce order status hook not triggering
- Execute a function on each element of an array
- Group a 2d array by year-month of a date column, add column of counts and create a subarray from a column of ids in each group