I struggling to configure response timeout for uwsgi and the documentation isn't helpful. I have pretty long queries with hard timeout on client side so I want to drop connection before it comes to flask or have some way to check if request sits in listen queue more than some threshold because no need to answer to request which will be knowingly dropped. Also I have CPU limitation so I want to exactly X workers work at same time. On current setup with locust I got 100% failure rate after some short period of time.
This is flask app
from flask import Flask, request
import requests
import json
import time
import uwsgi
app = Flask(__name__)
@app.route('/')
def test_request():
# This is always false
#if not uwsgi.is_connected(uwsgi.connection_fd()):
# print("Disconnected")
time.sleep(4) # Long query
return "Done"
if __name__ == "__main__":
app.run(host='0.0.0.0')
uwsgi config run with uwsgi uwsgi.ini
[uwsgi]
strict = true
module = flask:app
master = true
processes = 2
threads = 1
enable-threads = true
listen = 8
# I don't want to kill workers
#harakiri = 60
http-timeout = 5
need-app = true
lazy = true
# Not clear what the difference with http-timeout
http-connect-timeout = 5
http = 0.0.0.0:5000
locust config
from locust import HttpUser, between, task, constant
class MyWebsiteUser(HttpUser):
wait_time = constant(1)
@task
def load_main(self):
self.client.get("/", timeout=10) # Hard client timeout
I interpret results as uwsgi either doesn't respect timeouts or this config have nothing with request/response timeout and still trying to respond to dropped clients.
I'm not limited to uwsgi but it seems to be the most advanced.
The closest possible solution I found was from AWS Sagemaker example
Effectively solution uses nginx as rate limiter and gunicorn as load balancer. On my locust tests to avoid queue overflow, I had to add limit_req_zone with static key.
System throughput increased three order of magnitude, flask on its own is the paragon of unscalability.