Search code examples
pythongenerator

How to find the length of a generator containing a huge number of elements, whithout creating a list, which would take eons to be created?


To get the length of a generator, i.e. the number of elements it contains, we normally create a list from it and then the length of the list. However, there are times when a generator contains a huge amount of elements, and creating a list is an unacceptable method since it takes eons. Is there some other way to get the length of such a generator much faster?

(An example could be permutations of 10 by 10 with repetitions, the length of which 10^10 = 10000000000 --but we are not supposed to know this-- and creating a list from such a generator would take eons.)


A link has been suggested (What's the shortest way to count the number of items in a generator/iterator?) as a solution the the present question. The question of that link is "What's the shortest way to count the number of items in a generator/iterator?". The only common it has with the current question is "How to find the length of a generator"!

It is interesting to see that the second and most important part in the current question, which has to do with "huge number of elements" and "taking eons to create" has been totally ignored.

The classic len(list(gen)) or sum(1 for _ in gen) methods take eons to execute for a generator with n to the power of n items, where n > 8. And this is what the current question is all about.


Solution

  • A thing that immediately came to mind is:

    generator_instance = get_generator()
    length = sum(1 for _ in generator_instance)