How does pytest create fixtures, or, how to get my yield?

Question

I have a generator function

def foo():
    resource = setup()
    yield resource
    tidy(resource)

I'm using this as a fixture

@pytest.fixture
def foofix():
    yield from foo()

This works fine.

I want to test foo

def test_foo():
    res_gen = foo()
    res = next(res_gen)
    assert res.is_working

but because I tidy up resource immediately after I yield it, it's not longer available to assert it's working. How then does pytest use resource before it's tidied up, and how can I do the same?

Answer 1

If you want to test foo, you have just to call it in your code, not to wrap it in a pytest fixture.

Fixtures are good for values that require some boiler plate to be generated, and, when these values have to be tidyed up after the test, in a transparent way to the test - so that the test can consume the value, and do not worry about any boiler plate.

If you want to test the foo generator itself, and not just consume its value in a test, there is no sense in wrapping it in a fixture - simply import it, or its module, in the module containing the test function, start the generator by calling it, and then call next and catch StopIteration if needed.

When you run

def test_foo():
    res_gen = foo()
    res = next(res_gen)
    assert res.is_working

as a plain Python generator, after calling next on foo() you will get the resource before it is disposed: execution in the generator is paused at the yield resource expression when assert res.is_working is executed. Only upon the subsequent call to next would the tidy(resource) line run.

Actually this is another problem in your design: generators do not, in any way, ensure they will run to completion (and therefore finishing your resource) - in a pattern like this, it is up to the caller to call next until the generator is exhausted.

It is different when you mark a generator as a pytest fixture - in that case, pytest will run the generator again, after the first yield, after the test is gone - but that is a pytest "goodie", since it uses this pattern to start and to clean-up a fixture, not the natural behavior of generators.

The pattern there is close to it is to decorate such a generator with Python's contextlib.contextmanager call, and then your generator is transformed into a context manager that can be used as an expression in the with command. When the with command is over, the generator code bellow the yield is executed.

So, maybe you want this:

from contextlib import contextmanager

@contextmanager
def foo():
    resource = setup()
    # try/finally block is needed, otherwise, if an exception
    # occurs where resource is used in a `with` block, 
    # the finalization code is not executed.
    try: 
        yield resource
    finally:
        tidy(resource)

def test_foo():
    with foo() as res:
        # here, you have "resource" and it has not terminated
        assert res.is_working
    # the end of the "with" block resumes the generator and frees the resource
    return  # even if it is an implicit return.