I'm running into a problem wherein pytest_dependency works as expected when
EITHER
OR
But, I can't get the dependency to work properly when doing BOTH - parametrized dependent tests in a different file. It skips all the tests even when the dependencies have succeeded.
I have a directory structure like so:
tests/
- common.py
- test_0.py
- test_1.py
common.py:
import numpy as np
ints = [1, 2, 3]
strs = ['a', 'b']
pars = list(zip(np.repeat(ints, 2), np.tile(strs, 3)))
test_0.py:
import numpy as np
import pytest
from pytest_dependency import depends
from common import pars
def idfr0(val):
if isinstance(val, (int, np.int32, np.int64)):
return f"n{val}"
def idfr1(val):
return "n{}-{}".format(*val)
# I use a marker here because I have a lot of code parametrized this way
perm_mk = pytest.mark.parametrize('num, lbl', pars, ids=idfr0)
# 2 of these parametrized tests should fail
@perm_mk
@pytest.mark.dependency(scope="session")
def test_a(num, lbl):
if num == 2:
assert False
else:
assert True
# I set up a dependent parametrized fixture just like the in the documentation
@pytest.fixture(params=pars, ids=idfr1)
def perm_fixt(request):
return request.param
@pytest.fixture()
def dep_perms(request, perm_fixt):
depends(request, ["test_a[n{}-{}]".format(*perm_fixt)])
return perm_fixt
# This one works
@pytest.mark.dependency(scope="session")
def test_b(dep_perms):
pass
# These are non-parametrized independent tests
@pytest.mark.dependency(scope="session")
def test_1():
pass
@pytest.mark.xfail()
@pytest.mark.dependency(scope="session")
def test_2():
assert False
test_1.py:
import pytest
from pytest_dependency import depends
from common import pars
def idfr2(val):
return "n{}-{}".format(*val)
@pytest.fixture(params=pars, ids=idfr2)
def perm_fixt(request):
return request.param
@pytest.fixture()
def dep_perms(request, perm_fixt):
depends(request, ["test_0.py::test_a[n{}-{}]".format(*perm_fixt)])
return perm_fixt
# Same use of a parametrized fixture, but this one doesn't work
@pytest.mark.dependency(scope="session")
def test_c(dep_perms):
num, lbl = dep_perms
assert True
# These are non-parametrized dependent tests that work as expected
@pytest.mark.dependency(scope="session", depends=["test_0.py::test_1"])
def test_3():
pass
@pytest.mark.dependency(scope="session", depends=["test_0.py::test_2"])
def test_4():
pass
I expect to see test_a pass for 4 of its 6 parametrized runs and fail 2, test_b pass 4 and skip 2, and test_c likewise pass 4 and skip 2. I expect test_1 to pass, test_2 to xfail, test_3 to pass, and test_4 to be skipped. All of the above happens perfectly except for test_c - all of it gets skipped.
I've confirmed that the test names look like they are right. I run pytest from the tests directory like so:
pytest --tb=no -rpfxs ./test_0.py ./test_1.py
The output is:
collected 22 items
test_0.py ..FF....ss...x [ 63%]
test_1.py ssssss.s [100%]
=================================================================================== short test summary info ===================================================================================
PASSED test_0.py::test_a[n1-a]
PASSED test_0.py::test_a[n1-b]
PASSED test_0.py::test_a[n3-a]
PASSED test_0.py::test_a[n3-b]
PASSED test_0.py::test_b[n1-a]
PASSED test_0.py::test_b[n1-b]
PASSED test_0.py::test_b[n3-a]
PASSED test_0.py::test_b[n3-b]
PASSED test_0.py::test_1
PASSED test_1.py::test_3
FAILED test_0.py::test_a[n2-a] - assert False
FAILED test_0.py::test_a[n2-b] - assert False
XFAIL test_0.py::test_2
SKIPPED [1] test_0.py:36: test_b[n2-a] depends on test_a[n2-a]
SKIPPED [1] test_0.py:36: test_b[n2-b] depends on test_a[n2-b]
SKIPPED [1] test_1.py:20: test_c[n1-a] depends on test_0.py::test_a[n1-a]
SKIPPED [1] test_1.py:20: test_c[n1-b] depends on test_0.py::test_a[n1-b]
SKIPPED [1] test_1.py:20: test_c[n2-a] depends on test_0.py::test_a[n2-a]
SKIPPED [1] test_1.py:20: test_c[n2-b] depends on test_0.py::test_a[n2-b]
SKIPPED [1] test_1.py:20: test_c[n3-a] depends on test_0.py::test_a[n3-a]
SKIPPED [1] test_1.py:20: test_c[n3-b] depends on test_0.py::test_a[n3-b]
SKIPPED [1] ..\..\Miniconda3\envs\python_utils\Lib\site-packages\pytest_dependency.py:101: test_4 depends on test_0.py::test_2
===================================================================== 2 failed, 10 passed, 9 skipped, 1 xfailed in 0.43s ======================================================================
Notice that it explicitly states that (for example) test_0.py::test_a[n1-a] has passed, but later it skips test_c[n1-a] because it depends on test_0.py::test_a[n1-a]. Yet test_3 passes because test_1 passed, and test_4 is skipped because test_2 xfailed, so I know my root node name is right.
I've scoured the other issues here but the vast majority of them are from naming or scope issues, both of which don't appear to be a problem here. Can anybody tell me why test_c doesn't work?
I believe it's failing to find the dependency in the other file, because the depends() function uses scope='module' by default. Change that to
depends(request, ["test_0.py::test_a[n{}-{}]".format(*perm_fixt)], scope='session')
And the dependent tests work as expected.
What helped me in finding this issue was displaying the debug logs pytest-dependency emits, by passing --log-cli-level=debug when running pytest.