I am creating a flask app and running it on gunicorn, but when I use python's requests module it times out. Here is a minimal reproducible example:
from flask import Flask
import requests
API_URL = "http://127.0.0.1:8000"
def create_app():
app = Flask(__name__)
@app.route('/')
def index():
response = requests.get(API_URL + "/test")
return response
@app.route('/test')
def test():
return "test"
return app
I run the code with gunicorn "app.main:create_app()". When I just change the index function to a simple return of text, it works no problem. However, when I run it with gunicorn, I get the error
[2024-04-06 11:04:54 -0400] [45422] [INFO] Starting gunicorn 20.0.4
[2024-04-06 11:04:54 -0400] [45422] [INFO] Listening at: http://127.0.0.1:8000 (45422)
[2024-04-06 11:04:54 -0400] [45422] [INFO] Using worker: sync
[2024-04-06 11:04:54 -0400] [45423] [INFO] Booting worker with pid: 45423
[2024-04-06 11:05:28 -0400] [45422] [CRITICAL] WORKER TIMEOUT (pid:45423)
[2024-04-06 11:05:28 -0400] [45423] [INFO] Worker exiting (pid: 45423)
[2024-04-06 11:05:28 -0400] [45436] [INFO] Booting worker with pid: 45436
After the timeout happens, the /test route gets called. Does anyone know why this is the case? Thank you!
Overall, I tried to call the local /test endpoint with python's requests. I was expecting it to instantly go through and return "test". It instead gave me the timeout error above.
The problem is not requests with gunicorn. If you change the URL to an external server, then it will probably work and not timeout.
The problem here is that when you run gunicorn, by default it uses 1 worker only. So what's happening is that when you make a request to http://127.0.0.1:8000/ (index) the worker will handle that request and have no workers left to handle new requests until it finishes, but because inside index() you are making a request to your server, the worker will try to make a request to itself but there won't be workers available to handle that new request, creating a deadlock.
To fix it try running gunicorn with more than 1 worker. gunicorn --workers 2 "app.main:create_app()"