Search code examples
pythonbashdockercondauwsgi

uWSGI with conda env keeps failing (Docker)

I've been trying multiple approaches to get to deploy a Flask API with conda packages using uWSGI & NGINX without luck.

Would hugely appreciate any assistance to get this working.

In my current approach, I use NGINX base image, then install python packages, conda, create a new env with libraries part of the environment.yml. Then in my startup script, I activate the conda env, start nginx service, then uwsgi.

What I see happening is that uWSGI is failing to run as part of the conda env that has the packages installed and ends up failing due to imports. However, from the same place if I run python main.py it starts properly in development mode.

uwsgi.ini

[uwsgi]
module = main:app
uid = root
gid = root
master = true
processes = 5

socket = /tmp/uwsgi.socket
chmod-sock = 664
vacuum = true
plugins    = http,python3

die-on-term = true

Dockerfile

FROM nginx
SHELL [ "/bin/bash", "--login", "-c" ]

RUN apt-get update \
    && apt-get install -yyq netcat && apt-get install -y sudo


RUN apt-get update && apt-get install -y \
    python3-pip python3-dev uwsgi-plugin-python3 \
    nginx supervisor curl git wget

RUN apt-get install -y libpq-dev

COPY environment.yml requirements.txt /tmp/

COPY postBuild.sh /usr/local/bin/postBuild.sh
RUN chmod u+x /usr/local/bin/postBuild.sh
COPY docker/entrypoint.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/entrypoint.sh

COPY app /usr/local/bin/app
COPY app/nginx.conf /etc/nginx

RUN chmod u+x /usr/local/bin/app/start.sh

ENV MINICONDA_VERSION latest
ENV CONDA_DIR $HOME/miniconda3
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-$MINICONDA_VERSION-Linux-x86_64.sh -O ~/miniconda.sh && \
    chmod +x ~/miniconda.sh && \
    ~/miniconda.sh -b -p $CONDA_DIR && \
    rm ~/miniconda.sh
ENV PATH=$CONDA_DIR/bin:$PATH
RUN echo ". $CONDA_DIR/etc/profile.d/conda.sh" >> ~/.profile

RUN conda init bash
ENV PROJECT_DIR $HOME/app
WORKDIR $PROJECT_DIR
ENV ENV_PREFIX $PWD/env

RUN conda update --name base --channel defaults conda && \
    conda env create --prefix $ENV_PREFIX --file /tmp/environment.yml --force && \
    conda clean --all --yes

WORKDIR /usr/local/bin/app/



RUN conda activate $ENV_PREFIX && \
    /usr/local/bin/postBuild.sh && \
    conda deactivate


CMD ["./start.sh"]

start.sh

export PATH=/miniconda/bin:$PATH
echo ". /miniconda3/etc/profile.d/conda.sh" >> ~/.profile
conda init bash
eval "$(conda shell.bash hook)"
export PYTHONPATH=/env/bin/python
conda activate /env
service nginx start
uwsgi --ini uwsgi.ini
Solution

  • The problem is that you are installing two different versions of Python: one via apt-get (which is what uwsgi wants), and one using Conda.

    You should install uwsgi as a conda package, i.e. list it as a dependency in your environment.yml. It should be available via conda-forge: https://anaconda.org/conda-forge/uwsgi

    See https://pythonspeed.com/articles/importerror-docker/ for a checklist of things to check when Python programs won't run or libraries won't import.