How do Python Generators know who is calling?

Question

This question is making me pull my hair out.

if I do:

def mygen():
    for i in range(100):
        yield i

and call it from one thousand threads, how does the generator know what to send next for each thread?

Everytime I call it, does the generator save a table with the counter and the caller reference or something like that?

It's weird.

Please, clarify my mind on that one.

Answer 1

mygen does not have to remember anything. Every call to mygen() returns an independent iterable. These iterables, on the other hand, have state: Every time next() is called on one, it jumps to the correct place in the generator code -- when a yield is encountered, control is handed back to the caller. The actual implementation is rather messy, but in principle you can imagine that such an iterator stores the local variables, the bytecode, and the current position in the bytecode (a.k.a. instruction pointer). There is nothing special about threads here.