How to use an ARG variable in a RUN command of a Dockerfile?

I'm writing a Dockerfile, in which I want the python version and the requirements file to be configurable. It is as follows:

ARG VERSION=3.9
ARG REQUIREMENTS_FILE="../requirements.txt"

FROM python:$VERSION

RUN python3 -m ensurepip
RUN python3 -m pip install -r $REQUIREMENTS_FILE

CMD python3 -m pip freeze

However, when I build the image (docker build -t myimage:1.0 --build-arg VERSION=3.10 --build-arg REQUIREMENTS_FILE="../requirements.txt" .), I get the following error:

 => ERROR [3/3] RUN python3 -m pip install -r $REQUIREMENTS_FILE                                                                   0.6s
------                                                                                                                                  
 > [3/3] RUN python3 -m pip install -r $REQUIREMENTS_FILE:                                                                              
0.539                                                                                                                                   
0.539 Usage:                                                                                                                            
0.539   /usr/local/bin/python3 -m pip install [options] <requirement specifier> [package-index-options] ...                             
0.539   /usr/local/bin/python3 -m pip install [options] -r <requirements file> [package-index-options] ...                              
0.539   /usr/local/bin/python3 -m pip install [options] [-e] <vcs project url> ...
0.539   /usr/local/bin/python3 -m pip install [options] [-e] <local project path> ...
0.539   /usr/local/bin/python3 -m pip install [options] <archive url/path> ...
0.539 
0.539 -r option requires 1 argument
------
Dockerfile:7
--------------------
   5 |     
   6 |     RUN python3 -m ensurepip
   7 | >>> RUN python3 -m pip install -r $REQUIREMENTS_FILE
   8 |     
   9 |     CMD python3 -m pip freeze
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 -m pip install -r $REQUIREMENTS_FILE" did not complete successfully: exit code: 2

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/lgd2p5onypc74hsljarhv7a9o

The URL leads to the same error printed before.

I build this image from a shell script, but the error is the same from CLI . I tried with ${REQUIREMENTS_FILE} as well, and with an array: RUN ["python3", "-m", "pip", "install", "-r", $REQUIREMENTS_FILE], nothing worked. Is what I am trying to do even possible ?

Solution

Args are scoped to the stage where they are defined, and at the top of the file, they are only defined for the FROM lines. So you would need to move it down:

ARG VERSION=3.9

FROM python:$VERSION

ARG REQUIREMENTS_FILE="../requirements.txt"
RUN python3 -m ensurepip
RUN python3 -m pip install -r $REQUIREMENTS_FILE

CMD python3 -m pip freeze

Then you'll encounter the next issue, which is that the file doesn't exist inside the image. You'll want to learn about the build context and COPY command for more on that.

I tried with ... an array: RUN ["python3", "-m", "pip", "install", "-r", $REQUIREMENTS_FILE]

That will never work since docker doesn't expand build args in a RUN step, it just passes them as environment variables. You need a shell to expand environment variables, and the json array syntax, aka exec syntax, explicitly runs the step without a shell.