python generator function that is allowed to iterate over mutations

Question

I want to define a generator function that can iterate through a dictionary object that is mutating.

def generator(d):
    for k,v in sorted(d.items()):
        yield (k,v)

d = {1:'a', 2:'x', 4:'m', 8:'d', 16:'f'}
i = generator(d)
print(next(i))
print(next(i))
del d[8]
print(next(i))
d[16] = "f"
d[32] = "z"
print(next(i))
print(next(i))

If I execute the code above, I will get:

(1, 'a')
(2, 'x')
(4, 'm')
(8, 'd')
(16, 'f')

The desired output should be:

(1, 'a')
(2, 'x')
(4, 'm')
(16, 'f')
(32, 'z')

The part I don't understand is when I assign d = {1:'a', 2:'x', 4:'m', 8:'d', 16:'f'} to i = generator(d) The dictionary is already added into function generator(d) and becomes a generator instance with the initial dictionary. So whenever I use del function, the generator stays unchanged with d = {1:'a', 2:'x', 4:'m', 8:'d', 16:'f'} as its parameter. How am I gonna achieve this?

Answer 1

you need to deal with exceptions to check when the dictionary is changed and a buffer to keep the keys already yielded.

def generator(d):
    buffer = set()
    while True:
        old_hash = hash(frozenset(d.items()))
        try:
            for k, v in d.items():
                if hash(frozenset(d.items())) != old_hash:
                    raise RuntimeError('dictionary changed size during iteration')
                if (k, v) not in buffer:
                    buffer.add((k, v))
                    yield k, v
            break
        except RuntimeError as e:
            if str(e) != 'dictionary changed size during iteration':
                raise e

when the dictionary is modified the exception causes the for loop to restart, skipping over key-values pairs already yielded, the outer loop restarts everything when the dictionary is changed and breaks when every element in d has been yielded.

The only thing you need to be aware of is that when every item is returned and the generator stops, a dictionary change won't restart it the way the code is right now

EDIT added hashes for catching a very specific case mentioned in comments