Search code examples
pythonwebservertornadogunicornweb-frameworks

Common web servers/frameworks for deploying python machine learning models?


I noticed both Databricks MLFlow and Amazon Sagemaker both make use of the combination of Nginx, Gunicorn, and Flask.

I was wondering if that tends to be the standard stack for deploying machine learning models (e.g. Scikit-learn, Tensorflow, Keras).

If so, is there are reason those are used over alternatives like Tornado?

If not, is there a "standard stack"


Solution

  • I suppose, main reason for choosing a Flask over Tornado as a web-framework to wrap ML is that Flask's is simple but feature-rich (using addons). ML in general is CPU and RAM bound and the main advantage of the Tornado is effective handling of IO-bound operations, which is not the case for ML. But choosing Tornado for ML means more complex development for litte-to-zero benefit. It would be wiser if you have main app in Tornado (if you really need to) and use ML with Flask as a microservice to your main app.