Management long running tasks in Aiohttp app

Question

I have a web application in Aiohttp.

How to manage long running tasks? I see this scenario. Is this bad or good?

1. User makes request for some long running tasks.
2. Server create task with new_task = asyncio.create_task() Generate uuid for new task and save it all in dict:

new_task = asyncio.create_task()
uuid_task = uuid.uuid4()
tasks_set.update({
    uuid_task: new_task
})

3. Send answer to client with status 202 Accepted and task's uuid.
4. After some time user makes request with task uuid for asking task's status.
5. Server looks for in tasks_set for task and get it status:

task = tasks_set.get(uuid_from_client)
if not task:
    raise TaskNotFound # send error, 404 for example
if not task.done():
    # task is not done yet
    answer_to_client('task is not done')
    return
else:
    answer_to_client('task is done. Result: ', task.result())
    tasks_set.pop(uuid_from_client)

But I also have to manage timeout for tasks (user has gone and we should stop his task). Any suggestion?

Answer 1

But I also have to manage timeout for tasks

You can use asyncio.wait_for to add a timeout to any coroutine. Instead of:

# run coroutine in a new task
new_task = asyncio.create_task(coroutine(...))

you can use:

# run coroutine in a new task, for no longer than 10s
new_task = asyncio.create_task(asyncio.wait_for(coroutine(...), 10)

new_task.done() will be true both if the coroutine completes and if it times out. You can test for timeout by testing new_task.done() and new_task.exception() is asyncio.TimeoutError.