Search code examples
pythongeneratoryieldpep

return and return None in a generator: PEP guidelines


According to PEP 8 we should be consistent in our function declarations and ensure that they all have the same return-pattern, i.e. all should return an expression or all should not. However, I am not sure how to apply this to generators.

A generator will yield values as long as the code reaches them, unless a return statement is encountered in which case it will stop the iteration. However, I don't see any use-case in which returning a value from a generator function can happen. In that spirit, I don't see why it is useful - from a PEP 8 perspective - to end such a function with the explicit return None. In other words, why do we ought to verbalize a return statement for generators if the return expression is only reached when the yield'ing is over?

Example: in the following code, I don't see how hello() can be used to assign 100 to a variable (thus using the return statement). So why does PEP 8 expect us to write a return statement (be it 100 or None).

def hello():
    for i in range(5):
      yield i

    return 100


h = [x for x in hello()]
g = hello()

print(h)    
# [0, 1, 2, 3, 4]
print(g)
# <generator object hello at 0x7fd2f285a7d8>
# can we ever get 100?

Solution

  • You have misread PEP8. PEP8 states:

    Be consistent in return statements. Either all return statements in a function should return an expression, or none of them should.

    (bold emphasis mine)

    You should be consistent with how you use return within a single function, not across your whole project.

    Use return, it's the only return statement in the function.

    However, I don't see any use-case in which returning a value from a generator function can happen.

    The return value of a generator is attached to the StopIteration exception raised:

    >>> def gen():
    ...     if False: yield
    ...     return 'Return value'
    ...
    >>> try:
    ...     next(gen())
    ... except StopIteration as ex:
    ...     print(ex.value)
    ...
    Return value
    

    And this is also the mechanism by which yield from produces a value; the return value of yield from is the value attribute on the StopIteration exception. A generator can thus return a result to code using result = yield from generator by using return result:

    >>> def bar():
    ...     result = yield from gen()
    ...     print('gen() returned', result)
    ...
    >>> next(bar(), None)
    gen() returned Return value
    

    This feature is used in the Python standard library; e.g. in the asyncio library the value of StopIteration is used to pass along Task results, and the @coroutine decorator uses res = yield from ... to run a wrapped generator or awaitable and pass through the return value.

    So, from a PEP-8 point of view, for generators and there are two possibilities:

    • You are using return to exit the generator early, say in a loop with if. Use return, no need to add None:

      def foo():
          while bar:
              yield ham
              if spam:
                  return
      
    • You are using return <something> to exit and set StopIteration.value. Use return <something> consistently throughout your generator, even when returning None:

      def foo():
          for bar in baz:
              yield bar
              if spam:
                  return 'The bar bazzed the spam'
          return None