I am trying to process a file uploaded by the user. However, I want the user to get a response once the upload is completed and terminate the connection but carry on processing the file. My code looks something like this:
@app.route("/upload", methods=['POST'])
async def upload(request):
try:
file = request.files.get("file")
except Exception as e:
return json({"Received": False})
await process(file)
return response.text("File has been uploaded successfully")
async def process(file):
""" Processing File and generate data"""
...
result = await foo(data)
In the current framework, the user has to wait until the process completes. I would like the user to get a response saying:
"File has been uploaded successfully"
and terminate the connection but still continue with process(file).
I think you have two possible solutions here. The first one is going to slightly deviate from your requirements, because it will not terminate the connection. However, I think it is a viable solution, so I offer it as well. If not for your benefit, maybe for someone else that has a similar need.
This option is a streaming response where you send back the first part immediately, and then continue on processing with responses as needed. Now, this is obviously somewhat dependent upon how much processing we are talking about. If it is going to be somewhat long (pick an arbitrary time, 60s), then this solution may not work.
import asyncio
from functools import partial
from sanic import Sanic
from sanic.response import json, stream
app = Sanic(__name__)
async def process(file, response):
print(file, flush=True)
await response.protocol.push_data(b"0x0")
await response.write("File has been uploaded successfully\n")
await asyncio.sleep(2)
await response.write("33% processed\n")
await asyncio.sleep(2)
await response.write("66% processed\n")
await asyncio.sleep(2)
await response.write("100% processed\n")
@app.post("/")
async def upload(request):
try:
file = request.files.get("file")
except Exception:
return json({"Received": False})
return stream(partial(process, file), chunked=False)
I believe this is more what you are looking for. Immediate response (with a closed connection). We take the processing and push it off to the background with add_task.
import asyncio
from functools import partial
from sanic import Sanic
from sanic.response import json, text
app = Sanic(__name__)
async def process(file,):
await asyncio.sleep(3)
print(f"{file=}", flush=True)
@app.post("/")
async def upload(request):
try:
file = request.files.get("file")
except Exception:
return json({"Received": False})
request.app.add_task(process(file))
return text("File has been uploaded successfully")
As another alternative, here is an example that I put together where the handler pushes the task of to a queue, and there is a separate "worker" that is draining the queue and executing the tasks.