How to write complex `pytest` skip decorators?

Question

From the docs, it seems like the intended way to write skip decorators (that you'd import from e.g. conftest.py) is to use skipif (https://docs.pytest.org/en/6.2.x/skipping.html#id1). However, the condition shown in the example is simple and doesn't need to interact with any other parametrizations. Is it possible to get skipif to work when you also need to inspect arguments as part of the condition to skip?

In https://github.com/scipy/scipy/blob/4a6db1500dc62870865fe7827524abd332a88fd9/scipy/conftest.py#L147-L180, we have decorators which perform the skips we want correctly, however, (IIUC) because we use skip rather than skipif, when we use -rsx, the skips are reported as coming from conftest.py, e.g. SKIPPED [27] scipy/conftest.py:159: is_isomorphic only supports NumPy backend. Is there a way to write the decorators such that the skips are reported from the test in which they originate?

We can recover that information from --verbose instead, but it would be much easier if this worked with -rsx. Cheers!

# conftest.py
array_api_compatible = pytest.mark.parametrize("xp", xp_available_backends.values())

def skip_if_array_api_gpu(func):
    reason = "do not run with Array API on and not on CPU"
    # method gets there as a function so we cannot use inspect.ismethod
    if '.' in func.__qualname__:
        @wraps(func)
        def wrapped(self, *args, **kwargs):
            xp = kwargs["xp"]
            if SCIPY_ARRAY_API and SCIPY_DEVICE != 'cpu':
                if xp.__name__ == 'cupy':
                    pytest.skip(reason=reason)
                elif xp.__name__ == 'torch':
                    if 'cpu' not in torch.empty(0).device.type:
                        pytest.skip(reason=reason)
            return func(self, *args, **kwargs)
    else:
        @wraps(func)
        def wrapped(*args, **kwargs):
            # ditto
            return func(*args, **kwargs)
    return wrapped

# example test
@skip_if_array_api_gpu
@array_api_compatible
def test_xxx(xp):
   ...

x-ref https://github.com/pytest-dev/pytest/discussions/11726

Answer 1

We ended up using a fixture and a marker:

def pytest_configure(config):
    config.addinivalue_line("markers",
        "skip_if_array_api(*backends, reasons=None, np_only=False, cpu_only=False): "
        "mark the desired skip configuration for the `skip_if_array_api` fixture.")

@pytest.fixture
def skip_if_array_api(xp, request):
    if "skip_if_array_api" not in request.keywords:
        return
    backends = request.keywords["skip_if_array_api"].args
    kwargs = request.keywords["skip_if_array_api"].kwargs
    np_only = kwargs.get("np_only", False)
    cpu_only = kwargs.get("cpu_only", False)
    if np_only:
        reasons = kwargs.get("reasons", ["do not run with non-NumPy backends."])
        reason = reasons[0]
        if xp.__name__ != 'numpy':
            pytest.skip(reason=reason)
        return
    if cpu_only:
        reason = "do not run with `SCIPY_ARRAY_API` set and not on CPU"
        if SCIPY_ARRAY_API and SCIPY_DEVICE != 'cpu':
            if xp.__name__ == 'cupy':
                pytest.skip(reason=reason)
            elif xp.__name__ == 'torch':
                if 'cpu' not in torch.empty(0).device.type:
                    pytest.skip(reason=reason)
    if backends is not None:
        reasons = kwargs.get("reasons", False)
        for i, backend in enumerate(backends):
            if xp.__name__ == backend:
                if not reasons:
                    reason = f"do not run with array API backend: {backend}"
                else:
                    reason = reasons[i]
                pytest.skip(reason=reason)

Each test then needs the pytest.mark.usefixtures("skip_if_array_api") decorator, along with the pytest.mark.skip_if_array_api(...) decorator to choose the options. It would be nice to have just a single decorator rather than the two, but verbosity can be kept low by applying the first decorator to every test in a module using pytestmark = pytest.mark.usefixtures("skip_if_array_api")`.