Search code examples
pipcondaspacypythonanywhere

Deploying nlp algorithm with spaCy on pythonanywhere


I have a text classification algorithm that runs on spaCy and sklearn. I'm trying to deploy it via flask and pythonanywhere. The problem is I can't get my environments to match up.

I can install spaCy on my local machine, but only through Conda. I can install spaCy on pythonanywhere, but only through pip. The problem is that whith the conda install I get two packages:

mkl-fft==1.0.6
mkl-random==1.0.1

that apparently the pip installation doesn't use. So what I think is happening is that when I deploy the Conda trained algorithm to the pythonanywhere environment where spaCy was pip installed, it doesn't work. the two packages cannot be installed on pythonanywhere.

How do I solve this problem? Do I need to find a host that can do conda installs? If so which ones can accommodate this? Can I solve this by pip installing spaCy on my own machine and retraining the algorithm?

Not sure if any of this is clear, I'm just not sure how to proceed.


Solution

  • The problem here was that spaCy, numpy and scipy were configured to use MKL. In order to solve this I did create a new virtual environment and then did a conda install of the nomkl module. From there I just reinstalled the packages and they were no longer MKL configured and I was able to get consistent remote and local environments.