Moved from pip to poetry and now pytest-cov won't collect coverage data

Question

I recently started using poetry to manage project dependencies, rather than using requirements.txt and test-requirements.txt and pip.

Since making the change, I'm not able to get coverage tests to work correctly. In both cases, I'm using tox to drive the testing (and I have the tox-poetry extension installed).

My tox.ini currently looks like this:

[tox]
isolated_build = True
envlist = pep8,unit

[testenv]
whitelist_externals = poetry

[testenv:venv]
commands = {posargs}

[testenv:pep8]
commands =
    poetry run flake8 {posargs:symtool}

[testenv:unit]
commands =
    poetry run pytest --cov=symtool {posargs} tests/unit

Previously, it looked like this:

[tox]
envlist = pep8,unit

[testenv]
usedevelop = True
install_command = pip install -U {opts} {packages}
deps = -r{toxinidir}/requirements.txt
    -r{toxinidir}/test-requirements.txt

[testenv:venv]
commands = {posargs}

[testenv:pep8]
commands =
    flake8 {posargs:symtool}

[testenv:unit]
commands =
    pytest --cov=symtool {posargs} tests/unit

Since making the change to poetry, when I run e.g. tox -e unit, I see:

unit run-test: commands[0] | poetry run pytest --cov=symtool tests/unit
===================================== test session starts =====================================
platform linux -- Python 3.9.1, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
cachedir: .tox/unit/.pytest_cache
rootdir: /home/lars/projects/symtool, configfile: tox.ini
plugins: cov-2.11.1
collected 14 items

tests/unit/test_disasm.py .....                                                         [ 35%]
tests/unit/test_symtool.py .........                                                    [100%]r
Coverage.py warning: No data was collected. (no-data-collected)

I'm trying to figure out that no-data-collected issue. According to pytest --help, the --cov arguments set the path or package name:

 --cov=[SOURCE]        Path or package name to measure during execution

The root of the repository (which is rootdir in the above output from tox) looks like this:

asm
pyproject.toml
README.md
reference
symtool
tests
tox.ini

There's definitely a symtool directory there containing the package. And even if the tests were somehow not running in the project root directory, the symtool package is installed in the test environment, as evidenced by the fact that the unit tests are actually passing (all of which include some variant of import symtool).

How do I get coverage to work again?

Answer 1

Turning my comments into an answer:

This is an old issue with pytest-cov and tox (first reported in issue #38). tox installs your project as a third-party package and pytest will import it from the site (e.g. from .tox/unit/lib/python3.X/site-packages if the job is named unit), while --cov=symtool instructs pytest-cov to collect coverage over the symtool dir in project root.

One solution is to switch to the src layout:

├── pyproject.toml
├── README.md
├── reference
├── src
|   └── symtool
├── tests
└── tox.ini

In your pyproject.toml, you will need to point poetry to source dir:

[tool.poetry]
...
packages = [
    { include = "symtool", from = "src" },
]

Now src will prevent importing code from the source tree, so --cov=symtool will collect coverage over installed modules, this being the only option. For the rest of the dev process, you shouldn't get into trouble as poetry install installs the project in editable mode, so the rest should just work.

Another option is to skip package installation with tox and install in editable mode instead. Example snippet to place in tox.ini:

[testenv]
whitelist_externals =
    poetry
skip_install = true
commands_pre =
    poetry install
commands =
    poetry run pytest ...

This pretty much kills testing of the installed modules though (you stop testing explicitly whether your project can be installed in a blank venv, instead using what's in the source tree), so I'd go with the src layout option.