There are obviously similar questions a lot, but I have to ask this same thing again since I can not find a solution (which works) from the other questions (nor anywhere else with reasonable search).
Also note that I have just started with Python.
Also I feel it's needed to explain all that I currently have. So ... sorry for the long post
The Main problem
I have a legacy Python project that I would need to get to work locally (on Ubuntu 20.04). This project has environment.yml and requirements.txt files.
A virtual environment is created with:
conda env create -f environment.yml
This creates the virtual environment, which I can activate and see it has 'flask' installed.
conda list |grep flask
>> flask 1.1.2 pypi_0 pypi
pip list |grep Flask
>> flask 1.1.2 pypi_0 pypi
But when I attempt to run the application, I get the error:
ModuleNotFoundError: No module named 'flask'
The versions for the 'flask' were actually 1.1.2 for conda and 1.1.1 for pip at the beginning, but I changed these to be the same at some point (same error nevertheless, tried several other versions as well).
Main question: What am I doing/having wrong?
What I have and what I have tried
I tried doing the pip install by the requirements.txt after the environment was created. I did this both in- and outside of the virtual environment.
pip install -r requirements.txt
But this results in a lot of errors because of libraries not found, so I think it is perhaps out of date (or just doesn't work with my current environments). I understand it is supposed to represent the libraries installed for the environment where the application works. (My guess is that this is originally taken from the container running the app?)
I did find a solution that worked. I manually installed the flask (and several other libraries after that) with apt:
sudo apt install python3-flask
However this is obviously not the right solution as the libraries are supposed to be only in the virtual environment (afaik). Also it seems this worked once with pip (maybe with anaconda ... don't know anymore. Couldn't make it happen again.). I think I could maybe write these requirements by hand to a new "requirments_local.txt" and use that if I could the pip install work again but I think this would be wrong (why to have environment.yml and a requirements -file?). (the environment.yml has also pip libraries to install)
I don't think this is significant, but writing this down just in case.
The application is started via two bash-scripts where the 1st one sets up some runtime settings and then calls another script (entrypoint.sh which is used by the container also) which ultimately has this line to start the actual application:
exec gunicorn -b 0.0.0.0:5000 myapp:app
(Seems to be ok, since the app starts in a docker container and even locally when ?globally? installing the dependencies) The virtual environment is activated before calling the 2nd script, by having this line in the script:
source activate $env_name
I am using source since I noticed 'conda activate $env_name' did not work. I checked that the correct environment is in use by executing 'conda env list > conda.txt' in different stages at both scripts.
I use Ubuntu 20.04 and have these installed:
I have also tried using anaconda instead of miniconda since there seems to be some dependencies in requirements.txt pointing to anaconda, but the error with the flask was the same.
Additional questions: Can I omit the requirements.txt from the version control or can it be integrated somehow with the environment.yml? It does not feel right to have library versions listed in two files. What is even the purpose of the requirements when already creating the virtual environment?
Although suggestion by @razimbres was not actually correct, it gave me an idea to think for reasons why the libraries are not found from the virtual environment. (Could there be something that directs the app to look for libraries outside the env)
I had installed the green unicorn by apt since the absence of it was the first error when originally starting the app.
exec gunicorn -b 0.0.0.0:5000 myapp:app
./entrypoint.sh: line 62: exec: gunicorn: not found
Since it was missing from the environment.yml and not knowing any better, I installed it via apt (since it was there ... though someone could now explain why? It clearly just messes up things.).
I added to the environment.yml the following (conda) dependency:
- gunicorn==20.1.0
Then created the environment again:
conda env remove --name $env_name
conda env create -f environment.yml
And now it works. (even starting with the scripts)
I think the lesson was: Do not use apt for installing any python resources.
Also. If you had any, uninstall all python resources installed via apt. Like in my case:
sudo apt remove gunicorn
Ps. Also it seems that the requirements.txt is not needed. Maybe it can be removed (later). Feels it is just causing confusion at the moment.