Search code examples
pythonloopscombinationspython-itertools

Continue iteration in 3 dimensions i, j, k using itertools.combinations from certain value


I post this question since all existing answers solve this issue in one dimension using islice, which is not the same (at least I couldn't figure out how to transfer current solutions to three dimensions).

Suppose we have the following script that produces all discrete combinations of integers for three dimensions ranging from (1 to 127):

for j, k, l in itertools.combinations(range(1, 128), 3):
    result = calculations([128, i, j, k, 0])
    writeFile(result+"\n")

and suppose that the script gets interrupted at a random (recorded) point, e.g. [128, 121, 98, 45]. How can we use islice (or any other library) to continue with next iteration, i.e. [128, 121, 98, 46] and onwards until [128,127,126,125]?

Thank you for your time,


Solution

  • as a somewhat dirty hack, iter(f, sentinel) produces an iterator that repeatedly calls the given function until it returns the given sentinel value, so if you knew the last finished value you could pass combos.__next__ and the last given value to iter and exhaust that iterator to skip to the next viable item:

    import itertools, collections
    all_combos = iter(map(set, itertools.combinations(range(1, 128), 3)))
    LAST_RECORDED_COMBO = {121, 98, 45}
    ITER_WE_WANT_TO_SKIP = iter(all_combos.__next__, LAST_RECORDED_COMBO)
    # taken right from https://stackoverflow.com/questions/36763084/how-to-efficiently-exhaust-an-iterator-in-a-oneliner
    collections.deque(ITER_WE_WANT_TO_SKIP, maxlen=0)
    
    for combo in all_combos: # all_combos has now skipped forward
        print(combo) # original loop etc.
    

    The above is little more than than just looping through the combos stopping when it sees the last one used like this:

    for c in all_combos:
        if c == LAST_RECORDED_COMBO:
            break
    for combo in all_combos: # all_combos has now skipped forward
        print(combo) # original loop etc.
    

    But optimized so it can run entirely on the C layer much like islice to help performance.

    I used map(set, ..) here because the sentinel value must be equal which means the ordering of a list or tuple would matter for it to stop properly, otherwise this has really bad feedback since it will just exhaust the entire iterator without a clear indication of why.