Search code examples
pythondocker

Skip downloading files if the Docker build process is interrupted

I'm new to Docker and want to build a Dockerfile with the following snippet:

RUN \
  cur=`pwd` && \
  wget http://www.coppeliarobotics.com/files/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \
  tar -xf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \
  export COPPELIASIM_ROOT="$cur/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04" && \
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT:$COPPELIASIM_ROOT/platforms && \
  export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT && \
  git clone https://github.com/stepjam/PyRep.git && \
  cd PyRep && \
  pip3 install -r requirements.txt && \
  pip3 install setuptools && \
  pip3 install .

Because of the network issue, the build process is unstable and interrupted on the git clone... line. However, when I try to build again, CoppeliaSim_Edu_V4_1_0_Ubuntu20_04 which has already been saved is downloaded again, which consumes a lot of time. Is there any method to skip the download process if the .tar file has already been saved during the last build process?

Solution

  • Put the download command of .tar files on a separate command.

    RUN wget ttp://www.coppeliarobotics.com/files/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \
  tar -xf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz

RUN # rest of your code

    Each instruction in the Dockerfile translates to a layer in your final image. You can think of image layers as a stack, with each layer adding more content on top of the layers that came before it:

    image

    Whenever a layer changes, that layer will need to be re-built. If a layer changes, all other layers that come after it are also affected.

    source : docker docs

    What it means is:

    After the first built, Docker stores each layer in Cache, in the future builds, if the dockerfile content doesn't change, docker uses the cached layer instead of rebuilding them.

    To your question: "Is there any method to skip the download process if the .tar file has already been saved during the last build process?"

    Put the download .tar file command in a separate RUN Command that precedes your other commands to take advantage of docker caching.