Search code examples
pythongenerator

How to len(generator())


Python generators are very useful. They have advantages over functions that return lists. However, you could len(list_returning_function()). Is there a way to len(generator_function())?

UPDATE:
Of course len(list(generator_function())) would work.....
I'm trying to use a generator I've created inside a new generator I'm creating. As part of the calculation in the new generator it needs to know the length of the old one. However I would like to keep both of them together with the same properties as a generator, specifically - not maintain the entire list in memory as it may be very long.

UPDATE 2:
Assume the generator knows it's target length even from the first step. Also, there's no reason to maintain the len() syntax. Example - if functions in Python are objects, couldn't I assign the length to a variable of this object that would be accessible to the new generator?


Solution

  • Generators have no length, they aren't collections after all.

    Generators are functions with a internal state (and fancy syntax). You can repeatedly call them to get a sequence of values, so you can use them in loop. But they don't contain any elements, so asking for the length of a generator is like asking for the length of a function.

    if functions in Python are objects, couldn't I assign the length to a variable of this object that would be accessible to the new generator?

    Functions are objects, but you cannot assign new attributes to them. The reason is probably to keep such a basic object as efficient as possible.

    You can however simply return (generator, length) pairs from your functions or wrap the generator in a simple object like this:

    class GeneratorLen(object):
        def __init__(self, gen, length):
            self.gen = gen
            self.length = length
    
        def __len__(self): 
            return self.length
    
        def __iter__(self):
            return self.gen
    
    g = some_generator()
    h = GeneratorLen(g, 1)
    print len(h), list(h)