Search code examples
pythonmachine-learningflaskscikit-learn

Reaching maximum resources on python webhost

I want to deploy a flask app on a python webhost. The app is built using machine learning models link KNN, SVM, etc. I have developed the models using the scikit-learn library. The app works fine on the my local machine, but encounters Internal Server Error when I run it on the host. I checked the error log and found that when running the ML models, the server runs out of resources. How can I fix this?

I have tried limiting the n_jobs in scikit-learn models to 1, but it didn't work.

Solution

  • I encountered this issue two months ago and could resolved it after spending days on it.

    You can restrict the access of your scikit-learn model to the resources using joblib:

    from joblib import parallel_backend
import sklearn.neighbors.KNeighborsClassifier as KNN
model = KNN(n_jobs=1)
with parallel_backend("threading", n_jobs=1):
    model.fit(X,y)