next() doesn't play nice with any/all in python

Question

I ran down a bug today that came about because I was using next() to extract a value, and 'not found' emits a StopIteration.

Normally that would halt the program, but the function using next was being called inside an all() iteration, so the all just terminated early and returned True.

Is this an expected behavior? Are there style guides that help avoid this kind of thing?

Simplified example:

def error(): return next(i for i in range(3) if i==10)
error() # fails with StopIteration
all(error() for i in range(2)) # returns True

Answer 1

While this is the default behaviour in Python versions up to and including 3.6, it's considered to be a mistake in the language, and is scheduled to change in Python 3.7 so that an exception is raised instead.

As PEP 479 says:

The interaction of generators and StopIteration is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently, StopIteration raised accidentally inside a generator function will be interpreted as the end of the iteration by the loop construct driving the generator.

From Python 3.5 onwards, it's possible to change the default behaviour to that scheduled for 3.7. This code:

# gs_exc.py

from __future__ import generator_stop

def error():
    return next(i for i in range(3) if i==10)

all(error() for i in range(2))

… raises the following exception:

Traceback (most recent call last):
  File "gs_exc.py", line 8, in <genexpr>
    all(error() for i in range(2))
  File "gs_exc.py", line 6, in error
    return next(i for i in range(3) if i==10)
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "gs_exc.py", line 8, in <module>
    all(error() for i in range(2))
RuntimeError: generator raised StopIteration

In Python 3.5 and 3.6 without the __future__ import, a warning is raised. For example:

# gs_warn.py

def error():
    return next(i for i in range(3) if i==10)

all(error() for i in range(2))

$ python3.5 -Wd gs_warn.py 
gs_warn.py:6: PendingDeprecationWarning: generator '<genexpr>' raised StopIteration
  all(error() for i in range(2))

$ python3.6 -Wd gs_warn.py 
gs_warn.py:6: DeprecationWarning: generator '<genexpr>' raised StopIteration
  all(error() for i in range(2))