No module named 'Cython' setting up Azure ML docker instance

Question

I'm trying to install the following library in my Azure ML instance:

https://github.com/philferriere/cocoapi#egg=pycocotools&subdirectory=PythonAPI

My Dockerfile looks like this:

FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.0.3-cudnn8-ubuntu18.04:20210615.v1

ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/pytorch-1.7

# Create conda environment
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
    python=3.7 \
    pip=20.2.4 \
    pytorch=1.7.1 \
    torchvision=0.8.2 \
    torchaudio=0.7.2 \
    cudatoolkit=11.0 \
    nvidia-apex=0.1.0 \
    -c anaconda -c pytorch -c conda-forge

# Prepend path to AzureML conda environment
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH

# Install pip dependencies
RUN HOROVOD_WITH_PYTORCH=1 \
    pip install 'matplotlib>=3.3,<3.4' \
                'psutil>=5.8,<5.9' \
                'tqdm>=4.59,<4.60' \
                'pandas>=1.1,<1.2' \
                'scipy>=1.5,<1.6' \
                'numpy>=1.10,<1.20' \
                'azureml-core==1.31.0' \
                'azureml-defaults==1.31.0' \
                'azureml-mlflow==1.31.0' \
                'azureml-telemetry==1.31.0' \
                'tensorboard==2.4.0' \
                'tensorflow-gpu==2.4.1' \
                'onnxruntime-gpu>=1.7,<1.8' \
                'horovod[pytorch]==0.21.3' \
                'future==0.17.1' \
                'git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI'

# This is needed for mpi to locate libpython
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH

An error is thrown when the library is being installed:

  Cloning https://github.com/philferriere/cocoapi.git to /tmp/pip-install-_i3sjryy/pycocotools
[91m    ERROR: Command errored out with exit status 1:
     command: /azureml-envs/pytorch-1.7/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_i3sjryy/pycocotools/PythonAPI/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_i3sjryy/pycocotools/PythonAPI/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-o68by1_q
         cwd: /tmp/pip-install-_i3sjryy/pycocotools/PythonAPI
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-_i3sjryy/pycocotools/PythonAPI/setup.py", line 2, in <module>
        from Cython.Build import cythonize
    ModuleNotFoundError: No module named 'Cython'

I've tried adding Cython as a dependecy in both the pip section and as part of the conda environment but the error is still thrown.

Answer 1

Solution was to add the following to the Dockerfile:

# Install Cython
RUN pip3 install Cython

# Install pip dependencies
RUN HOROVOD_WITH_PYTORCH=1 \
    pip install 'matplotlib>=3.3,<3.4' \
                ...
                'git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI'