keras prediction gets stuck when deployed using uwsgi in a flask app

Question

I have a keras model that works perfectly in unit tests and in local flask app (flask run). However, the moment I launch the flask app in uwsgi, it gets stuck from the second request on, killing the entire app. Is this because uwsgi spawns multiple processes? How do I get around this problem? Thanks.

Answer 1

I am having the same issue, this may not be ideal work around. But I believe this is an Keras issue running in multiple processes.

I am running uWSGI with lazy-apps = true.

uwsgi --http 0.0.0.0:5000 --wsgi-file your_flask_server.py --callable app --processes 2 --threads 2 --stats 127.0.0.1:9191 --lazy-apps

Note: By lazy loading apps this will consume more memory like double the memory since it is loading the whole app again in each thread/process

here are some use full links: Similar Issue, Similar Issue

Example deployment guide