How to estimate the iteration time and display it with tqdm of a for loop?

Question

I want to estimate the duration of the iterations of the following code and display it on a progress bar (tdqm). This function will give all the possible combinations of the string.printable.

---

import strings
from itertools import combinations

def generate_combinations(iterable):
  lst = list(iterable)
  for lenght in range(0, len(lst) + 1):
    for i in combinations(lst, lenght):
      with open('passwords.txt', 'a') as f:
        w = ''.join(map(str, i))
        f.write(w + '\n')

generate_combinations(string.printable)

---

I think you can do this using time and tqdm.

Expected output:

estimated time: 12 seconds left
tqdm progress bar: 12% |||||||...

Answer 1

The final answer (with the suggestion by @jq170727) is the following code:

---

import string
from tqdm.auto import trange, tqdm
from itertools import permutations

The requested generator (on google colab, using TPU: around 10k it / s)

def combined(lst, lenStart, lenEnd):
  for i in trange(lenStart, lenEnd + 1, desc='Progress'):
    for subset in tqdm(permutations(lst, i),  desc='Subsets', leave = False):
      w = ''.join(map(str, subset))
      yield w

---

And here we got the same generator with better efficiency (on google colab, using TPU: around 950k / s)

def compress_permutation(lst, lenStart, lenEnd):
  yield ','.join([''.join(map(str, subset)) for i in trange(lenStart, lenEnd + 1, desc='Progress') for subset in tqdm(permutations(lst, i), desc='Subsets', leave = False)])

iterable = list(string.printable[:94])
lenStart, lendEnd = 1, 9

for subset in compress_permutation(iterable, l1, l2):
  with open('passwords.txt', 'a') as f:
    f.write(subset)