Search code examples
dockerdockerfile

Replacing character in dockerfile variable - error: missing ':' in substitution


In my docker file I have a variable defined for a version number with dots which I want to replace with underscores for a further usage.

ARG ABC_VERSION=1.2.3
ARG SOME_OTHER_VARIABLE=/dir_name/abc_${ABC_VERSION//./_}

Unfortunately, I get the following error.

failed to process "/dir_name/abc_${ABC_VERSION//./_}": missing ':' in substitution

I need the version number several times within the dockerfile multiple times with the '.' and one time with the '_' and I do not like to define two variables.

Does someone know how to solve this problem?


Edit: One portion of the actual code where I would like to make use of the feature to replace characters is looking like this.

ARG EXPAT_VERSION=2.1.0
# ...
RUN wget https://github.com/libexpat/libexpat/releases/download/R_${EXPAT_VERSION//./_}/expat-${EXPAT_VERSION}.tar.gz \
    && tar xzf expat-${EXPAT_VERSION}.tar.gz \
    && cp -R expat-${EXPAT_VERSION}/lib ./xmp_sdk/third-party/expat \
    && rm -r expat-${EXPAT_VERSION} && rm expat-${EXPAT_VERSION}.tar.gz

I saw that something similiar is used in the tensorflow-gpu dockerfiles:

ARG CUDA=10.1
#...
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    cuda-command-line-tools-${CUDA/./-} #...

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/dockerfiles/dockerfiles/devel-gpu.Dockerfile


Solution

  • The ${parameter/pattern/string} syntax is actually a Bash feature (cf. shell parameter expansion), not a POSIX feature.

    According to the official documentation, the Dockerfile directives only supports:

    $var
    ${var}
    ${var:-word} → if var is not set then word is the result;
    ${var:+word} → if var is set then word is the result, otherwise the empty string
    

    Workaround 1

    So the problem does not have a "direct" solution, but if the variable you would like to substitute will be used, in the end, in some shell command (in a RUN, ENTRYPOINT or CMD directive), you could just as well keep the initial value as is (with no substitution), then substitute it later on?

    I mean for example, the following Dockerfile:

    FROM debian
    
    ARG ABC_VERSION=1.2.3
    ENV SOME_OTHER_VARIABLE=/app/${ABC_VERSION}
    
    WORKDIR /app
    
    RUN /bin/bash -c 'touch "${SOME_OTHER_VARIABLE//./_}"'
    
    # RUN touch "${SOME_OTHER_VARIABLE//./_}"
    # would raise /bin/sh: 1: Bad substitution
    
    CMD ["/bin/bash", "-c", "ls -hal \"${SOME_OTHER_VARIABLE//./_}\""]
    

    As an aside:

    • I replaced ARG SOME_OTHER_VARIABLE with ENV SOME_OTHER_VARIABLE just to be able to use it from CMD.
    • It can be recalled that ENTRYPOINT and CMD directives should rather be written in exec form − CMD ["…", "…"] − rather in shell form (see e.g. that question: CMD doesn't run after ENTRYPOINT in Dockerfile).

    Workaround 2

    Or as an alternative workaround, you may want to split your version number in major, minor, patch, to write something like this?

    ARG MAJOR=1
    ARG MINOR=2
    ARG PATCH=3
    ARG ABC_VERSION=$MAJOR.$MINOR.$PATCH
    ARG SOME_OTHER_VARIABLE=/dir_name/abc_${MAJOR}_${MINOR}_${PATCH}
    …
    

    A more concise syntax for workaround 1

    Following the OP's edit, I guess one concern is the relative verbosity of this line that I mentioned in the "workaround 1":

    …
    RUN /bin/bash -c 'touch "${SOME_OTHER_VARIABLE//./_}"'
    

    To alleviate this, Docker allows one to replace the implied shell (by default sh) with Bash, which does support the shell parameter expansion you are interested in. The key point is the following directive that has to be written before the RUN command (and which was precisely part of the Dockerfile the OP mentioned):

    SHELL ["/bin/bash", "-c"]
    

    Thus, the Dockerfile becomes:

    …    
    ARG ABC_VERSION=1.2.3
    
    SHELL ["/bin/bash", "-c"]
    RUN touch "/dir_name/abc_${ABC_VERSION//./_}" \
      && ls -hal "/dir_name/abc_${ABC_VERSION//./_}"
    

    or taking advantage of some temporary environment variable:

    …    
    ARG ABC_VERSION=1.2.3
    
    SHELL ["/bin/bash", "-c"]
    RUN export SOME_OTHER_VARIABLE="/dir_name/abc_${ABC_VERSION//./_}" \
      && touch "$SOME_OTHER_VARIABLE" \
      && ls -hal "$SOME_OTHER_VARIABLE"