Search code examples
pythonrangegeneratorstring-interning

Does python use interning on generator objects? Weird behaviour with (i for i in range(n))


Why do these two loops not give the same result? (Yes, I know the second version is bad style. But I'd still expect it to give the same output.)

gen1 = range(3)
gen2 = range(3)
print("First attempt")
for i in gen1:
  for j in gen2:
    print(i,j)

gen1 = (i for i in range(3))
gen2 = (i for i in range(3))
print("Second attempt")
for i in gen1:
  for j in gen2:
    print(i,j)

Output using Python 3.6.9 (and I get the same results with Python 2.7.15):

First attempt
(0, 0)
(0, 1)
(0, 2)
(1, 0)
(1, 1)
(1, 2)
(2, 0)
(2, 1)
(2, 2)
Second attempt
(0, 0)
(0, 1)
(0, 2)

Solution

  • range is its own type. It’s iterable, but not an iterator (or generator) – you can iterate over it multiple times, each time creating independent iterators:

    >>> r = range(2)
    >>> list(r)
    [0, 1]
    >>> list(r)
    [0, 1]
    >>> next(r)
    TypeError: 'range' object is not an iterator
    

    Generators, like from generator expressions, are iterators. When you iterate over them, you advance them.

    >>> r = (1 + x for x in range(2))
    >>> list(r)
    [1, 2]
    >>> list(r)
    []
    
    >>> r = (1 + x for x in range(2))
    >>> iter(r) is r
    True
    >>> next(r)
    1
    

    And, to answer the first question explicitly, this behaviour isn’t related to interning. During the first iteration of the for i in gen1 loop, gen2 is consumed, so the remaining iterations do nothing.