Results Are Getting Mixed Up For Concurrent Requests in FastApi Using Gunicorn Server

Question

As per the thread title, given the following API:

app = FastAPI()

app.add_middleware(
    CORSMiddleware,
    allow_origins=['*'],
    allow_credentials=True,
    allow_methods=['*'],
    allow_headers=['*'],
)

app.mount("/app/static", StaticFiles(directory="app/static"), name="static")


@app.post("/")
async def get_text(image: UploadFile = File(...)):
    
    temp_file = _save_file_to_disk(image, path="app/static/", save_as="temp")
    text = await ocr(temp_file)
   
    return {"text": text}

and the following Gunicorn command to fire up the application:

CMD ["gunicorn", "app:app", "-w", "4", "--threads", "12", "--timeout", "120", 
"--bind", "0.0.0.0:8080", "-k","uvicorn.workers.UvicornWorker", "--worker-connections", "1000"]

if 3 separate requests are sent concurrently by 3 different users, the results of the first user overtake the results of the second and third user

Answer 1

While you're missing relevant context in your question I'm going to attempt a guess:

temp_file = _save_file_to_disk(image, path="app/static/", save_as="temp")

This seems to save the uploaded file as temp under app/static. You then run ocr on this file.

text = await ocr(temp_file)

However - this file name will be the same for all three processes, meaning that the last request will overwrite the file with the content of request three. Each process will start their ocr-ing, but the file has been changed underneath the process.

Instead use a unique name for each file - and you should probably also keep it outside of anything you serve directly through static.