Search code examples
pythonfor-looppython-itertools

Python skipping for-loop after print()


I'm at debugging part of writing my code, so I'm using lots of calls to print to check the values of my variables. I have encountered a situation, when printing a variable causes Python to skip a for loop.

import numpy as np
import itertools as itr
(...)

class myclass():
    def a_function_inside_a_class():
        print('-------------')
        current_size = self.get_current_size()
        current_size[dimension] += 1

        new_layer = [range(i) for i in current_size]
        new_layer[dimension] = [current_size[dimension]]
        print(new_layer)

        new_layer = itr.product(*new_layer)
        print(list(new_layer)) # THIS LINE CAUSES PROBLEMS

        for c in new_layer:
            print('for')
            print(list(c))
            (...)
            a_function_that_causes_some_error(c)

I create a list, then use itertools to create combinations of this list and then iterate over them. If I call the a_function_inside_a_class() as it is above, I don't get the for printed. No error occurs. The interpreter does not enter the loop.

(...)
-------------
[[2], range(0, 1), range(0, 1)]
[(2, 0, 0)]
-------------
[range(0, 1), [2], range(0, 1)]
[(0, 2, 0)]
(...)

But if I comment out #print(list(new_layer)) then the for-loop is executed and I get an error caused by some other function a_function_that_causes_some_error(c).

-------------
[[2], range(0, 1), range(0, 1)]
for
[2, 0, 0]
Traceback (most recent call last):
(...)

The list(new_layer) does not change the new_layer object itself, only creates a list and passes it to print() function, does it?


Solution

  • The problem is that itertools.product returns a generator.

    When you call print(list(new_layer)) you are constructing a list from the generator, but don't save a reference to that list anywhere.

    The generator itself will be exhausted after converting it to a list, because list(some_generator) calls __next__ (or next, depending on Python version) on that generator until it raises StopIteration.

    >>> from itertools import product
    >>> new_layer = product([1,2,3], [4,5,6])
    >>> new_layer
    <itertools.product object at 0x7f46e90349b0>
    >>> print(list(new_layer))
    [(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
    >>> print(list(new_layer))
    []
    

    As you can see, after constructing the list once, the generator is empty.

    You can fix your program by using

    new_layer = list(itr.product(*new_layer))
    print(new_layer)
    

    because now you will have a reference to the list you are creating from the generator.