Search code examples
pythongeneratoryieldserver-sent-events

How can I yield notifications and return a result from a function? (Python)


I have to create a function that does some hard work in the internal calls. This function needs to be a generator because I'm using Server-Sent Events. So that, I want that this function notifies the progress of the calculations by using a "yield". After that, this function has to pass the result to the parent function in order to continue with other calculations.

I would like something like this:

def hardWork():
    for i in range(N):
        # hard work
        yield 'Work done: ' + str(i)

    # Here is the problem: I can't return a result if I use a yield
    return result             

def generator():
    # do some calculations
    result = hardWork()
    # do other calculations with this result
    yield finalResult

I found a solution that consists on yields a dictionary that tells if the function has finished or not, but the code to do this is pretty dirty.

Is there another solution?

Thank you!

EDIT

I thought something like:

def innerFunction(gen):
    calc = 1

    for iteration in range(10):
        for i in range(50000):
            calc *= random.randint(0, 10)
        gen.send(iteration)

    yield calc


def calcFunction(gen):
    gen2 = innerFunction(gen)
    r = next(gen2)

    gen.send("END: " + str(r + 1))
    gen.send(None)


def notifier():
    while True:
        x = yield
        if x is None:
            return
        yield "Iteration " + x


def generator():
    noti = notifier()
    calcFunction(noti)
    yield from noti


for g in generator():
    print(g)

But I'm receiving this error:

TypeError: can't send non-None value to a just-started generator

Solution

  • Prior to Python3.5: generators

    This solution also works for more recent Python versions, although async def, new in Python3.5, seems more fit for your usecase. See next section.

    The values yielded by a generator are obtained by iterating or using next. The value returned at the end is stored in the value attribute of the StopIteration exception that indicates the end of the generator. Fortunately, it is not too hard to recover.

    def hardWork():
        output = []
    
        for i in range(10):
            # hard work
            yield 'Doing ' + str(i)
            output.append(i ** 2)
    
        return output
    
    def generator():
        # do some calculations
        work = hardWork()
    
        while True:
            try:
                print(next(work))
            except StopIteration as e:
                result = e.value
                break
    
        yield result
    

    Example

    foo = generator()
    print(next(foo))
    

    Output

    Doing 0
    Doing 1
    Doing 2
    Doing 3
    Doing 4
    Doing 5
    Doing 6
    Doing 7
    Doing 8
    Doing 9
    [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
    

    Python3.5+: async def

    If you are running Python3.5+, what you are attempting seems perfectly fit for an event loop using awaitable functions.

    import asyncio
    
    async def hardWork():
        output = []
    
        for i in range(10):
            # do hard work
            print('Doing ', i)
            output.append(i**2)
    
            # Break point to allow the event loop to do other stuff on the side
            await asyncio.sleep(0)
    
        return output
    
    async def main():
        result = await asyncio.wait_for(hardWork(), timeout=None)
        print(result)
    
    loop = asyncio.get_event_loop()
    
    loop.run_until_complete(main())
    

    Output

    Doing  0
    Doing  1
    Doing  2
    Doing  3
    Doing  4
    Doing  5
    Doing  6
    Doing  7
    Doing  8
    Doing  9
    [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]