Search code examples
pythoncode-coveragepytestpytest-cov

Python Coverage turns top-level package name into period

I have a project laid out like this:

proj/ex_secure/__init__.py
proj/ex_secure/base.py
proj/ex_secure/metrics.py
proj/ex_secure/keys.py
proj/tests/test_base.py
proj/tests/test_metrics.py
proj/tests/test_keys.py
proj/.gitignore
proj/.pep8
proj/README.rst
proj/setup.cfg
proj/setup.py

If I run pytest like this:

pytest -s --junitxml=pytests.xml --cov-report xml --cov-report term-missing --cov-branch --cov=ex_secure

Then the top-level package in the resulting coverage.xml is named .:

<package branch-rate="0.4722" complexity="0" line-rate="0.6801" name=".">

But if I call pytest like this:

pytest -s --junitxml=pytests.xml --cov-report xml --cov-report term-missing --cov-branch --cov=ex_secure.base --cov=ex_secure.metrics --cov=ex_secure.keys

Then the top-level package is correctly named ex_secure:

<package branch-rate="0.4722" complexity="0" line-rate="0.6801" name="ex_secure">

It's an okay workaround for now, but it's not ideal. If I add more packages, I have to keep enumerating them (or they'll be missing on the coverage report). Additionally, __init__.py isn't covered using this mechanism.

What am I doing wrong here?

UPDATE 1:

If I run Python Coverage directly instead of using pytest-cov, it works as expected:

coverage run --branch --source=ex_secure -m pytest -s --junitxml=pytests.xml
coverage xml

Then:

<package branch-rate="0.4722" complexity="0" line-rate="0.6771" name="ex_secure">

UPDATE 2:

If I run PyTest as I originally did, but then re-generate the XML report with Python Coverage directly, the re-generated report gets corrected, but has slightly different numbers:

pytest -s --junitxml=pytests.xml --cov-report xml --cov-report term-missing --cov-branch --cov=ex_secure
coverage xml

Then:

<package branch-rate="0.3058" complexity="0" line-rate="0.4769" name="ex_secure">
Solution

  • Unfortunately, this is a rather nasty two-part bug that spans both Py-Coverage and PyTest-Cov. It is described in detail on the PyTest-Cov GitHub and the Py-Coverage BitBucket.

    In short, calling coverage xml from the command line doesn't allow you to pass in sources, so most people don't see this problem, but you can pass in sources when using the Python API for Py-Coverage, and that API does not correctly handle the sources attribute. PyTest-Cov, meanwhile, uses the Python API for Py-Coverage, so when you call PyTest-Cov with --cov=xxxx and --cov-report xml, you end up with this problem.