I have the following code:
def my_zip(*iterables):
iterators = tuple(map(iter, iterables))
while True:
yield tuple(map(next, iterators))
When my_zip is called, it just creates an infinite loop and never terminates. If I insert a print statement (like shown below), it is revealed that my_zip is infinitely yielding empty tuples!
def my_zip(*iterables):
iterators = tuple(map(iter, iterables))
while True:
t = tuple(map(next, iterators))
print(t)
yield t
However, the equivalent code with a generator expression works fine:
def my_genexp_zip(*iterables):
iterators = tuple(iter(it) for it in iterables)
while True:
try:
yield tuple(next(it) for it in iterators)
except:
print("exception caught!")
return
Why is the function with map not behaving as expected? (Or, if it is expected behavior, how could I modify its behavior to match that of the function using the generator expression?)
I am testing with the following code:
print(list(my_genexp_zip(range(5), range(0, 10, 2))))
print(list(my_zip(range(5), range(0, 10, 2))))
The two pieces of code you provided are not actually "equivalent", with the function using generator expressions notably having a catch-all exception handler around the generator expression producing items for tuple output.
And if you actually make the two functions "equivalent" by removing the exception handler:
def my_listcomp_zip(*iterables):
iterators = tuple(iter(it) for it in iterables)
while True:
yield tuple(next(it) for it in iterators)
print(list(my_listcomp_zip(range(5), range(0, 10, 2))))
you'll get a traceback of:
Traceback (most recent call last):
File "test.py", line 4, in <genexpr>
yield tuple(next(it) for it in iterators)
~~~~^^^^
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "test.py", line 6, in <module>
print(list(my_listcomp_zip(range(5), range(0, 10, 2))))
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "test.py", line 4, in my_listcomp_zip
yield tuple(next(it) for it in iterators)
~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: generator raised StopIteration
So it is clear by now that the reason why your infinite loop with while True: can end at all with your generator expression version of the function is because a RuntimeError is caught by your catch-all exception handler, which returns from the function.
And this is because since Python 3.7, with the implementation of PEP-479, StopIteration raised inside a generator gets automatically turned into a RuntimeError in order not to be confused with the StopIteration raised by an exhausted generator itself.
If you try your code in an earlier Python version (such as 2.7), you'll find the generator expression version of the function gets stuck in the infinite loop just as well, where the StopIteration exception raised by next bubbles out from the generator and gets handled by the tuple constructor to produce an empty tuple, just like the map version of your function. And addressing this exception masking effect is exactly why PEP-479 was proposed and implemented.