Search code examples
dockerdockerfile

Creating a docker container with a specific compiler version requires update-alternatives to be added as the last line of the Dockerfile

I would like to create a docker container which by default uses a specific compiler version:

ARG UBUNTU_VERSION=20.04
FROM ubuntu:${UBUNTU_VERSION}

ARG COMPILER_VERSION=10
ARG PETSC_VERSION=v3.18.6

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y software-properties-common
RUN add-apt-repository ppa:ubuntu-toolchain-r/test
RUN apt-get update
RUN apt-get install -y gcc-${COMPILER_VERSION} g++-${COMPILER_VERSION} gfortran-${COMPILER_VERSION}

RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-${COMPILER_VERSION} 60

RUN apt-get install -y build-essential git python3 openssh-server openssh-client python3-pip ffmpeg libsm6 libxext6 cmake pkg-config \
                       openmpi-bin libopenmpi-dev

RUN python3 -m pip install pytest numpy h5py pathlib requests pandas matplotlib plotly scikit-learn PyYAML Pillow vtk types-PyYAML pytest-cov

RUN echo "Using PETSC_VERSION: ${PETSC_VERSION}"
RUN git clone -b ${PETSC_VERSION} https://gitlab.com/petsc/petsc.git /petsc && \
    cd /petsc && \
    ./configure \
        PETSC_ARCH=linux-gnu \
        --with-fc=gfortran --with-cc=gcc --with-cxx=g++ \
        --with-fortran-bindings \
        --download-openmpi \
        --with-mpi-f90module-visibility=1 \
        --download-fftw \
        --download-hdf5 \
        --download-hdf5-fortran-bindings \
        --download-fblaslapack \
        --download-scalapack \
        --download-ml \
        --download-zlib && \
    make all

ENV PETSC_DIR /petsc
ENV PETSC_ARCH linux-gnu

ENV OMPI_ALLOW_RUN_AS_ROOT 1
ENV OMPI_ALLOW_RUN_AS_ROOT_CONFIRM 1

RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-${COMPILER_VERSION} 60

This works fine and when we run the container interactively, it will select the correct compiler version by default. However, we have to use the same update-alternatives for this purpose, once to ensure that our other packages are installed with the correct compiler, and once again in the end to ensure that anything that is executed in the container uses the specified versions as well. If we omit the last part, the originally present compiler will be used. Why is this happening, and is there a more efficient way we can design our Dockerfile?

Solution

  • 🚨 For working Dockerfile see end of answer. Here's some of the reasoning for how I got there.

    I think you'll find that you are not actually using GCC 10 to compile the PETSc code.

    Try inserting RUN gcc --version immediately before you clone and build the PETSc repository. I think you'll find that you see something like this:

    Step 13/13 : RUN gcc --version
 ---> Running in dc6e2b3c0095
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    This means that, despite you setting an alternative version of the compiler prior to that it's still reverting to the default compiler. Why?

    The reason for this is that something in the subsequent apt-get install is reverting the compiler change. If you change the order of operations to this then you'll find that GCC 10 is in effect when you come to compile PETSc.

    RUN apt-get install -y build-essential git python3 openssh-server openssh-client python3-pip ffmpeg libsm6 libxext6 cmake pkg-config \
                       openmpi-bin libopenmpi-dev

RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-${COMPILER_VERSION} 60

    Now you should see this before the clone/compile step:

    Step 13/13 : RUN gcc --version
 ---> Running in 77abf17d52bc
gcc (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    But now there's a snag: there's an error in the PETSc build. It prefers the default compiler version!

    My suggestion is to simply remove the first set of update-alternatives commands because they are not having any effect on the subsequent build.

    Anyway, here's a complete Dockerfile that should do what you want without duplication. I had to install an updated CMake.

    ARG UBUNTU_VERSION=20.04
FROM ubuntu:${UBUNTU_VERSION}

ARG COMPILER_VERSION=10
ARG PETSC_VERSION=v3.18.6

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get install -y \
        software-properties-common \
        gnupg \
        wget \
        lsb-release
RUN add-apt-repository ppa:ubuntu-toolchain-r/test

# For recent CMake version.
#
RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc | gpg --dearmor - | tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null
RUN echo "deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/kitware.list

RUN apt-get update
RUN apt-get install -y \
        gcc-${COMPILER_VERSION} \
        g++-${COMPILER_VERSION} \
        gfortran-${COMPILER_VERSION} \
        build-essential \
        git \
        python3 \
        openssh-server \
        openssh-client \
        python3-pip \
        ffmpeg \
        libsm6 \
        libxext6 \
        cmake \
        pkg-config \
        openmpi-bin \
        libopenmpi-dev

RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-${COMPILER_VERSION} 60 && \
    update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-${COMPILER_VERSION} 60

RUN python3 -m pip install pytest numpy h5py pathlib requests pandas matplotlib plotly scikit-learn PyYAML Pillow vtk types-PyYAML pytest-cov

RUN echo "Using PETSC_VERSION: ${PETSC_VERSION}"
RUN git clone -b ${PETSC_VERSION} https://gitlab.com/petsc/petsc.git /petsc && \
    cd /petsc && \
    ./configure \
        PETSC_ARCH=linux-gnu \
        --with-fc=gfortran --with-cc=gcc --with-cxx=g++ \
        --with-fortran-bindings \
        --download-openmpi \
        --with-mpi-f90module-visibility=1 \
        --download-fftw \
        --download-hdf5 \
        --download-hdf5-fortran-bindings \
        --download-fblaslapack \
        --download-scalapack \
        --download-ml \
        --download-zlib && \
    make all

ENV PETSC_DIR /petsc
ENV PETSC_ARCH linux-gnu

ENV OMPI_ALLOW_RUN_AS_ROOT 1
ENV OMPI_ALLOW_RUN_AS_ROOT_CONFIRM 1