Configure/Setting/Activate a Python Conda environment in Docker

I am trying to set up a conda environment inside a docker container, but it always keep failing at the environment creation stage in the dockerfile with the error: conda activate" fails because shell is not initialized

Below is my Dockerfile:

FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-runtime

WORKDIR /app

# SET BASH AS CURRENT SHELL
RUN chsh -s /bin/bash
SHELL ["/bin/bash", "-c"]

# INSTALL ANACONDA
RUN apt-get update -y

RUN apt-get install -y wget && \
    apt-get clean


RUN rm -rf /opt/conda && \
    wget --quiet https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh -O ~/anaconda.sh && \
    /bin/bash ~/anaconda.sh -b -p /opt/conda && \
    rm ~/anaconda.sh && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    find /opt/conda/ -follow -type f -name '*.a' -delete && \
    find /opt/conda/ -follow -type f -name '*.js.map' -delete && \
    /opt/conda/bin/conda clean -afy

COPY ./requirements.yml /tmp/requirements.yml


RUN conda update conda \
    && conda env create -f /tmp/requirements3.yml

RUN echo "conda activate nib_docker" >> ~/.bashrc
ENV PATH /opt/conda/envs/nib_docker/bin:$PATH
ENV CONDA_DEFAULT_ENV $nib_docker

COPY . .
CMD [ "python", "-u", "train.py" ]

Solution

Finally figured out:

To create a docker environment with its corresponding dependencies(both conda and pip), you can follow the below steps:

Create YML file

Create the yml file, which lists the conda environment details (conda and pip). You can export an existing conda environment in your local machine to a yml file using the below command Here, I'm exporting my myconda environment after activating it

conda activate myconda
conda env export --no-builds > env_config.yml
# --no-builds --> added to make sure that the build versions are not part of the final yml file as the build versions differ based on your base OS (Linux, Windows,macOS).

The YML file would look something like this:

name: myconda #rename this if you want to change the name of enviroment in docker
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - pip
  - conda_package1=4.0
  - conda_package2=5.0
  - conda_package3=6.0
  - pip:
    - pip_package1==6.0
    - pip_package2==7.0
    - pip_package3==8.0
    
prefix: C:\Users\ABC\.conda\envs\myconda

I prefer to separate my conda and pip dependencies into two separate files so that if I want to change any dependency, I'll change the respective file only to avoid download all packages again when creating the docker image. So, the two files will now look like this after editing:

env_config.yml

channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - pip
  - conda_package1=4.0
  - conda_package2=5.0
  - conda_package3=6.0
    
# prefix: C:\Users\ABC\.conda\envs\myconda  # not needed

env_pip_config.txt

pip_package1==6.0
pip_package2==7.0
pip_package3==8.0

And finally, the Dockerfile:

Dockerfile

## BASE IMAGE
FROM python:3.8

## SET WORKING DIRECTORY
WORKDIR /app

RUN apt-get update -y \
    && apt-get install -y wget \
    && apt-get clean
    
## CONDA INSTALLATION --> use the latest Anaconda version for linux from their official website. Google it buddy.
RUN rm -rf /opt/conda && \
    wget --quiet https://repo.anaconda.com/archive/Anaconda3-2023.01-0-Linux-x86_64.sh -O ~/anaconda.sh && \
    /bin/bash ~/anaconda.sh -b -p /opt/conda && \
    rm ~/anaconda.sh && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    find /opt/conda/ -follow -type f -name '*.a' -delete && \
    find /opt/conda/ -follow -type f -name '*.js.map' -delete && \
    /opt/conda/bin/conda clean -afy
    
## ADD CONDA PATH TO LINUX PATH 
ENV PATH /opt/conda/bin:$PATH

## COPY ENV REQUIREMENTS FILES
COPY ./env_config.yml /tmp/env_config.yml
COPY ./env_pip_config.txt /tmp/env_pip_config.txt


## CREATE CONDA ENVIRONMENT USING YML FILE
RUN conda update conda \
    && conda env create -f /tmp/env_config.yml


## Henceforth use your env name instead of "myconda" below

## ADD CONDA ENV PATH TO LINUX PATH 
ENV PATH /opt/conda/envs/myconda/bin:$PATH
ENV CONDA_DEFAULT_ENV myconda

## MAKE ALL BELOW RUN COMMANDS USE THE NEW CONDA ENVIRONMENT
RUN echo "conda activate myconda" >> ~/.bashrc


## INSTALL PIP DEPENDENCIES
RUN pip install -r /tmp/env_pip_config.txt

## COPY REST OF THE FILES
COPY . .

## FINALLY TIME TO EXECUTE!
CMD ["python", "myscript.py"]