Search code examples
powershelldockerfileenvironment-variables

powershell: How to pass environment variable in RUN command

I am trying to find a solution to the following simple problem. I would like to run the following cmake command:

> cmake "-DPKG_VERSION=$Env:PKG_VERSION" ..

It appears to be the correct syntax for this task (per doc).

The trouble is that this syntax does not work from within my Dockerfile:

Here are my failed attempt and the result I see:

[...]
Step 20/34 : RUN echo "-DPKG_VERSION="$($Env:PKG_VERSION)
 ---> Running in b02cebc8e124
-DPKG_VERSION=$
0.6.9
 ---> Removed intermediate container b02cebc8e124
 ---> 479fe1a125e8
Step 21/34 : RUN echo "-DPKG_VERSION="$Env:PKG_VERSION
 ---> Running in 8fc29b675b90
-DPKG_VERSION=$Env:
PKG_VERSION
 ---> Removed intermediate container 8fc29b675b90
 ---> 66372d417c9f
Step 22/34 : RUN echo "-DPKG_VERSION="${Env:PKG_VERSION}
 ---> Running in a014a9490f0a
-DPKG_VERSION=$
Env:PKG_VERSION
 ---> Removed intermediate container a014a9490f0a
 ---> 35befba98602
Step 23/34 : RUN echo "-DPKG_VERSION=$($Env:PKG_VERSION)"
 ---> Running in 03d126ef5aa5
-DPKG_VERSION=$
0.6.9
 ---> Removed intermediate container 03d126ef5aa5
 ---> 464ca62cf985
Step 24/34 : RUN echo "-DPKG_VERSION=$Env:PKG_VERSION"
 ---> Running in 34c83b2b01df
-DPKG_VERSION=$Env:
PKG_VERSION
 ---> Removed intermediate container 34c83b2b01df
 ---> 632bc7a489da
Step 25/34 : RUN echo "-DPKG_VERSION=${Env:PKG_VERSION}"
 ---> Running in da1640fbd40f
-DPKG_VERSION=$
Env:PKG_VERSION
 ---> Removed intermediate container da1640fbd40f
 ---> 341a14ec1552
Step 26/34 : RUN echo '-DPKG_VERSION=$Env:PKG_VERSION'
 ---> Running in f4038c2feb51
-DPKG_VERSION=$Env:PKG_VERSION
 ---> Removed intermediate container f4038c2feb51
 ---> e209c03cb636
Step 27/34 : RUN echo '-DPKG_VERSION=$($Env:PKG_VERSION)'
 ---> Running in c57df5ce803c
-DPKG_VERSION=$($Env:PKG_VERSION)

Question: what is the right way for passing environment variable in powershell (inside a Dockerfile) ?

Update: Minimal, Reproducible Example:

> docker build envvar
Sending build context to Docker daemon  2.048kB
Step 1/8 : ARG WSCI_VERSION=ltsc2019
Step 2/8 : FROM mcr.microsoft.com/windows/servercore:${WSCI_VERSION}
 ---> c632661e39bb
Step 3/8 : SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 ---> Using cache
 ---> 6b5e34b83047
Step 4/8 : ENV PKG_VERSION=1.2.3
 ---> Using cache
 ---> 6419bdf10920
Step 5/8 : RUN $PSVersionTable.PSVersion
 ---> Running in 02cdd6ef5360

Major  Minor  Build  Revision
-----  -----  -----  --------
5      1      17763  5122


Removing intermediate container 02cdd6ef5360
 ---> 97b514422293
Step 6/8 : RUN echo PKG_VERSION=${Env:PKG_VERSION}
 ---> Running in cb0de097efd9
PKG_VERSION=1.2.3
Removing intermediate container cb0de097efd9
 ---> ff8bc0534c61
Step 7/8 : RUN echo "-DPKG_VERSION=$Env:PKG_VERSION"
 ---> Running in 392ffea7a55b
-DPKG_VERSION=$Env:
PKG_VERSION
Removing intermediate container 392ffea7a55b
 ---> b66d9765535a
Step 8/8 : RUN echo "-DPKG_VERSION=${Env:PKG_VERSION}"
 ---> Running in 2f8f4d33983f
-DPKG_VERSION=$
Env:PKG_VERSION
Removing intermediate container 2f8f4d33983f
 ---> 6f22a75f80e8
Successfully built 6f22a75f80e8

with

> cat .\envvar\Dockerfile
# escape=`
ARG WSCI_VERSION=ltsc2019
FROM mcr.microsoft.com/windows/servercore:${WSCI_VERSION}

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

ENV PKG_VERSION=1.2.3

RUN $PSVersionTable.PSVersion
RUN echo PKG_VERSION=${Env:PKG_VERSION}
RUN echo "-DPKG_VERSION=$Env:PKG_VERSION"
RUN echo "-DPKG_VERSION=${Env:PKG_VERSION}"
Solution

  • On Windows, Docker doesn't automatically perform the \-escaping of " characters that powershell.exe, the Windows PowerShell CLI, requires in code passed to its -Command (-c) parameter.

    • Note: The problem does not arise on Unix-like platforms, where there's no process-level command line and arguments are passed as an array of verbatim tokens.

    Therefore, on Windows, be sure to manually \-escape " chars. that are part of your command:[1]

    RUN echo \"-DPKG_VERSION=$Env:PKG_VERSION\"
RUN echo \"-DPKG_VERSION=${Env:PKG_VERSION}\"

    As an aside:

    • The curious symptoms that resulted without " escaping stem from a known parameter-binder bug in PowerShell, where an - in effect - unquoted argument such as -DPKG_VERSION=$Env:PKG_VERSION is unexpectedly split into two arguments at the first : - see GitHub issue #6360.

    • A simple way to reproduce this on Windows is to run the following from cmd.exe:

      :: Run from cmd.exe
powershell.exe -Command echo "-FOO=$ENV:OS"
      • PowerShell's initial CLI command-line parsing removes the the " instances because they're unescaped (see below for background information), so that what ultimately executes is echo -FOO=$Env:OS, which triggers the bug and emits two strings, verbatim -FOO=$Env: and verbatim OS.
    Background information on \ vs. `-escaping in PowerShell:

    Given that inside a PowerShell session it is `, the so-called backtick, that serves as the escape character, it may be surprising that on the CLI command line \ must be used to escape " chars in -Command arguments.[1]

    Presumably, this was done to align with most other CLIs, which either require or also support this kind of "-escaping and to be able to use standard techniques for command-line parsing. Note that escaping " as \" is an additional, preliminary layer of escaping solely for the PowerShell CLI's command-line parsing:

    • Unescaped " are assumed to simply protect arguments from the calling shell and are removed during command-line parsing.

    • Escaped " (\") are assumed to be part of the PowerShell code to execute via -Command in then-unescaped form - where they may or may not have syntactic function. (In -File invocations, \" is assumed to be a verbatim " char. inside a pass-through argument).

    In short: unescaped " are removed, escaped " are preserved as ", and the result - possibly stitched together with spaces from what were technically multiple arguments on the command line - is then interpreted as PowerShell code, where the usual syntax and escaping rules apply.

    Notably, this means that `-escaping may additionally be needed to satisfy PowerShell's syntax rules:

    • If an (originally escaped) " appears inside an (originally escaped) "..." string; e.g., from cmd.exe:
    :: From cmd.exe - note the `\" sequences.
:: Prints: 'Nat "King" Cole'
powershell.exe -Command "echo \"Nat `\"King`\" Cole\""``</sup>
    • In shell-free invocations,[2] such as from automation tools (including when building a Dockerfile) and scheduled tasks, where passing multiline arguments to the PowerShell CLI is possible (without needing to also satisfy the line-continuation syntax of a calling shell), ` is situationally still needed for line-continuations, as shown in your own answer:
    # Part of your Dockerfile on Windows, assuming # escape=` is in effect.
# Note that *both* \-escaping (\")
# *and* `-escaping (line-continuations) are needed:
RUN echo "-Dfirst" `
         \"-DPKG_VERSION=${Env:PKG_VERSION}\" `
         "-Dthird"

    Note that a # escape=` parser directive in a Dockerfile allows Docker to recognize PowerShell's line continuations as such, so that their use to spread RUN instructions across multiple lines doesn't run afoul of Docker's parsing.

    The directive does not otherwise affect the content passed to the target shell, except that Docker actually removes the line continuations behind the scenes and passes a single-line representation to the shell.

    It is therefore technically possible - though perhaps confusing - to not use this directive - and use \ for line-continuation (even though PowerShell wouldn't understand it, it never gets to see it).

    [1] pwsh.exe, the PowerShell (Core) 7+ CLI, alternatively also (robustly) accepts "" in lieu of \". "" can help avoid running afoul of cmd.exe's command-line parsing, but in shell-free invocations of the PowerShell CLI - such as performed during a Docker build behind the scenes - \" works robustly.
    In PowerShell CLI invocations with the -File parameter (invoking a .ps1 script file), no interpretation of the arguments as PowerShell code occurs, and \" is only needed to embed verbatim " chars. in pass-through arguments.

    [2] By that I mean that the PowerShell CLI is invoked directly (ultimately via a WinAPI call), not from another shell.