Search code examples
pythonpython-2.7asynchronoustornado

How can I raise an exception through Tornado coroutines incorrectly called?


I have a scenario with Tornado where I have a coroutine that is called from a non-coroutine or without yielding, yet I need to propagate the exception back.

Imagine the following methods:

@gen.coroutine
def create_exception(with_yield):
    if with_yield:
        yield exception_coroutine()
    else:
        exception_coroutine()

@gen.coroutine
def exception_coroutine():
    raise RuntimeError('boom')

def no_coroutine_create_exception(with_yield):
    if with_yield:
        yield create_exception(with_yield)
    else:
        create_exception(with_yield)

Calling:

try:
    # Throws exception
    yield create_exception(True)
except Exception as e:
    print(e)

will properly raise the exception. However, none of the following raise the exception :

try:
    # none of these throw the exception at this level
    yield create_exception(False)
    no_coroutine_create_exception(True)
    no_coroutine_create_exception(False)
except Exception as e:
    print('This is never hit)

The latter are variants similar to what my problem is - I have code outside my control calling coroutines without using yield. In some cases, they are not coroutines themselves. Regardless of which scenario, it means that any exceptions they generate are swallowed until Tornado returns them as "future exception not received."

This is pretty contrary to Tornado's intent, their documentation basically states you need to do yield/coroutine through the entire stack in order for it to work as I'm desiring without hackery/trickery.

I can change the way the exception is raised (ie modify exception_coroutine). But I cannot change several of the intermediate methods.

Is there something I can do in order to force the exception to be raised throughout the Tornado stack, even if it is not properly yielded? Basically to properly raise the exception in all of the last three situations?

This is complicated because I cannot change the code that is causing this situation. I can only change exception_coroutine for example in the above.


Solution

  • What you're asking for is impossible in Python because the decision to yield or not is made by the calling function after the coroutine has finished. The coroutine must return without raising an exception so it can be yielded, and after that it is no longer possible for it to raise an exception into the caller's context in the event that the Future is not yielded.

    The best you can do is detect the garbage collection of a Future, but this can't do anything but log (this is how the "future exception not retrieved" message works)