I am attempting to set up a continuous integration environment using mypy, pytest, flake8, pytest-cov, and black. My directory structure for this test project titled hello is the following.
hello
|_hello
| |___init__.py
| |_add.py
|_tests
| |___init__.py
| |_test_add.py
|_pyproject.toml
my pyproject.toml file has the following content.
[tool.poetry]
name = "hello"
version = "0.1.0"
description = ""
authors = ["Jon Webb <[email protected]>"]
readme = "README.rst"
[tool.poetry.dependencies]
python = "^3.10"
[tool.poetry.group.dev.dependencies]
pytest = "^7.2.1"
flake8 = "^6.0.0"
mypy = "^1.0.1"
black = "^23.1.0"
isort = "^5.12.0"
flake8-bandit = "^4.1.1"
flake8-bugbear = "^23.2.13"
flake8-builtins = "^2.1.0"
flake8-comprehensions = "^3.10.1"
flake8-implicit-str-concat = "^0.4.0"
flake8-print = "^5.0.0"
tox = "^4.4.6"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
classifiers = [
"Development Status :: 4 - Beta",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"License :: OSI Approved :: MIT License",
"Operating System :: MacOS",
"Operating System :: POSIX :: Linux",
]
[tool.isort]
profile = 'black'
[tool.black]
line-length = 90
target-version = ['py38', 'py39', 'py310']
include = '\.pyi?$'
exclude = '''
/(
\.eggs
| \.git
| \.hg
| \.mypy_cache
| \.tox
| \.venv
| _build
| buck-out
| build
| dist
# The following are specific to Black, you probably don't want those.
| blib2to3
| tests/data
| profiling
)/
'''
[tool.tox]
legacy_tox_ini = """
[tox]
env_list = py310, coverage, mypy, flake8
[testenv]
description = run tests with pytest
deps = pytest
commands = pytest -v {posargs}
[testenv:coverage]
deps = pytest-cov
commands =
pytest --cov --cov-report term-missing --cov-report html --cov-append --cov-context=test --cov-fail-under=100 {posargs}
[testenv:mypy]
deps = mypy
commands = mypy hello
[testenv:flake8]
deps = flake8
commands = flake8 hello
"""
[tool.mypy]
exclude = ["docs", "tests"]
check_untyped_defs = true
disallow_any_generics = true
disallow_incomplete_defs = true
disallow_untyped_defs = true
no_implicit_optional = true
[tool.pytest.ini_options]
testpaths = ["tests"]
console_output_style = "progress"
When I run tox, everything works perfectly, except for code coverage. However, the code coverage is 100% as expected, and the problems appears to be something different than the coverage itself. I get the following output message, which seems to indicate that the testenv:coverage section of my config file is mangling with the testenv:mypy section. I get the following message when I run tox.
platform linux -- Python 3.10.9, pytest-7.2.1, pluggy-1.0.0
cachedir: .tox/coverage/.pytest_cache
rootdir: /home/jonwebb/Code_Dev/Python/hello, configfile: pyproject.toml, testpaths: tests
plugins: cov-4.0.0
collected 2 items
tests/test_add.py .. [100%]
---------- coverage: platform linux, python 3.10.9-final-0 -----------
Name Stmts Miss Cover Missing
-------------------------------------------------
conftest.py 0 0 100%
hello/__init__.py 0 0 100%
hello/add.py 4 0 100%
tests/__init__.py 0 0 100%
tests/test_add.py 12 0 100%
-------------------------------------------------
TOTAL 16 0 100%
Coverage HTML written to dir htmlcov
Required test coverage of 100% reached. Total coverage: 100.00%
============================== 2 passed in 0.03s ===============================
coverage: commands[1]> '[testenv:mypy]'
coverage: exit 2 (0.00 seconds) /home/jonwebb/Code_Dev/Python/hello> '[testenv:mypy]'
coverage: FAIL ✖ in 1.39 seconds
mypy: install_package> python -I -m pip install --force-reinstall --no-deps /home/jonwebb/Code_Dev/Python/hello/.tox/.tmp/package/51/hello-0.1.0.tar.gz
mypy: OK ✔ in 1.17 seconds
flake8: install_package> python -I -m pip install --force-reinstall --no-deps /home/jonwebb/Code_Dev/Python/hello/.tox/.tmp/package/52/hello-0.1.0.tar.gz
flake8: commands[0]> flake8 hello
.pkg: _exit> python /home/jonwebb/Code_Dev/Python/hello/.venv/lib/python3.10/site-packages/pyproject_api/_backend.py True poetry.core.masonry.api
py310: OK (1.33 seconds)
coverage: FAIL code 2 (1.39=setup[1.20]+cmd[0.19,0.00] seconds)
mypy: OK (1.17 seconds)
flake8: OK (1.28=setup[1.20]+cmd[0.08] seconds)
evaluation failed :( (5.29 seconds)
Does anyone know what might be causing this failure?
I am not sure why, but for some reason tox expects the coverage command to be last. When I re-write the command as the following, everything works fine.
[tool.tox]
legacy_tox_ini = """
[tox]
env_list = py310, coverage, mypy, flake8
[testenv]
description = run tests with pytest
deps = pytest
commands = pytest -v {posargs}
[testenv:mypy]
deps = mypy
commands = mypy hello
[testenv:flake8]
deps = flake8
commands = flake8 hello
[testenv:coverage]
deps = pytest-cov
commands =
pytest --cov --cov-report term-missing --cov-report html --cov-append --cov-context=test --cov-fail-under=100 {posargs}
"""