Search code examples
dockerfilecommand-substitution

How do I use command substition in Dockerfile

In my Dockerfile I need to use command substition to add some environment variables. I want to set

ENV PYTHONPATH /usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')

but it doesn't work. The result is 

foo@bar:~$ echo $PYTHONPATH
/usr/local/$(python3 -c from distutils import sysconfig; print(sysconfig.get_python_lib()))

What's wrong?

Solution

  • What went wrong

    The $( ... ) command substitution you attempted is for Bash, whereas the Dockerfile is not Bash. So docker doesn't know what to do with that, it's just plain text to docker, docker just spews out what you wrote as-is.

    Recommendation

    To avoid hard-coding values into a Dockerfile, and instead, to dynamically change a setting or custom variable as PYTHONPATH during the build, perhaps the ARG ... , --build-arg docker features might be most helpful, in combination with ENV ... to ensure it persists.

    Within your Dockerfile:

    ARG PYTHON_PATH_ARG

ENV PYTHONPATH ${PYTHON_PATH_ARG}

    In Bash where you build your container:

    python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"

docker build --build-arg PYTHON_PATH_ARG=$python_path .

    Explanation

    According to documentation, ARG:

    The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag.

    So, in Bash we first:

    python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"
    • $(...) Bash command substitution is used to dynamically put together a Python path value
    • this value is stored temporarily in a Bash variable $python_path for clarity
    docker build --build-arg PYTHON_PATH_ARG=$python_path .
    • Bash variable $python_path value is passed to docker's --build-arg PYTHON_PATH_ARG

    Within the Dockerfile:

    ARG PYTHON_PATH_ARG
    • so PYTHON_PATH_ARG stores the value from --build-arg PYTHON_PATH_ARG...

    ARG variables are not equivalent to ENV variables, so we couldn't merely do ARG PYTHONPATH and be done with it. According to documentation about Using arg variables:

    ARG variables are not persisted into the built image as ENV variables are.

    So finally:

    ENV PYTHONPATH ${PYTHON_PATH_ARG}
    • We use Dockerfile's ${...} convention to get the value of PYTHON_PATH_ARG, and save it to your originally named PYTHONPATH environment variable

    Differences from original code

    You originally wrote:

    ENV PYTHONPATH /usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')

    I re-wrote the Python path finding portion as a Bash command, and tested on my machine:

    $ python_path="/usr/local/$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"

$ echo $python_path
/usr/local//usr/lib/python3/dist-packages

    Notice there is a double forward slash ... local//usr ... , not sure if that will break anything for you, depends on how you use it in your code.

    Instead, I changed it to:

    $ python_path="/usr/local$(python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())')"

    Result:

    $ echo $python_path
/usr/local/usr/lib/python3/dist-packages

    So this new code will have no double forward slashes.