Search code examples
pythongeneratoryield

Adding item back into an iterable (yield/generator)


I thought this is a great moment to use yield, but I'm stuck.

When something fails, I would like to send the item back into the generator. I've read that this is possible, so I'd really like to use my first generator.

states = ["IL", "NY", "NJ"]
for state in states:
    ok = do something
    if not ok:
        *add state back as the first-to-deal with in the generator*

How to use a generator in such a case?


Solution

  • You are probably referring to a coroutine, which leverages the yield expression. It works a little like this:

    def co_gen(li):
        for x in li:
            bad = yield x
            if bad is not None:
                print('ack! {}'.format(bad))
                #other error handling
    

    and (contrived) usage:

    states = ["IL", "NY", "NJ"]
    
    gen = co_gen(states)
    
    for x in gen:
        print('processing state: {}'.format(x))
        if x == 'NY':
            y = gen.send('Boo, Yankees!')
            print( 'processing state after error: {}'.format(y))
    
    # processing state: IL
    # processing state: NY
    # ack! Boo, Yankees!
    # processing state after error: NJ
    

    Salient points - normal yield behavior assigns None to bad. If it's not None, something has been send-ed into the generator.

    When we send something into the generator, it resumes operation until it reaches the next yield expression. So keep that in mind - the above control flow in the coroutine isn't what I'd call "standard" since there is no yielding done in the error block.

    Here is a coroutine that operates a little more like what you were talking about:

    def co_gen(li):
        for x in li:
            bad = yield x
            while bad is not None:
                print('error in generator: {}'.format(bad))
                yield
                bad = yield bad
    
    gen = co_gen(states)
    
    for x in gen:
        print('processing state: {}'.format(x))
        if random.choice([0,1]):
            gen.send(x) #discard first yield
            print( 'error: trying {} again'.format(x) )
    
    # processing state: IL
    # error in generator: IL
    # error: trying IL again
    # processing state: IL
    # processing state: NY
    # error in generator: NY
    # error: trying NY again
    # processing state: NY
    # processing state: NJ
    

    We send our state back into the generator, and it keeps yielding it until we stop sending it.