Search code examples
powershellcmdescapingampersandquoting

Escaping the and (&) sign in Powershell

I need to use \r\n inside of ffmpeg --header parameter. This works in Unix, but not in Windows command prompt. So I was wondering to use powershell 

powershell c:\ffmpeg -headers 'User-Agent: user-agent'"`r`n"'Cookies: my-cookie' ...

I understand I have to use 'string' when special characters are used and 

"`r`n"

as my \r\n delimiter.

I have also tested that I can mix them together, like 'this '"is "'text' to get this is text

However if my string (cookie or user-agent) contains & character, it fails.

Example: powershell c:\ffmpeg -headers 'Cookies: param=a=1&b=2; session=123'

How can I "escape" & character in one-liner?

These examples (some parts are masked) are accepted by CMD, but they are not working

powershell -c "c:\ffplay -user_agent ""Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0"" -headers ""Cookie: nlqptid=h=676886edeea5bae904b0cf53daec8038.1519940538670&hdnea=exp=1519940598~acl=/*~hmac=C0E019502B060D23AB02BB157FCFFC72404500770A2CE5B00789A84AAEFBD77F&nltid=xxxxxxxxxx&nltdt=0&uid=744826&csid=xxxxxxxxxx; hdntl=exp=1520026938~acl=%2f*~hmac=952689e6de57a2a201ddc1d4c0794962fdc886ea48cf41494a42e787ef923bf9`r`n"" -i ""https://xxxxxxxxxx.net/nlds_vod/xxxxxxxxxx/vod/2018/02/28/cd264e10-dd54-3de2-df90-4c1cac104dcb/v1/stream/cd264e10-dd54-3de2-df90-4c1cac104dcb_1_4500_pc.mp4.m3u8"""

or

powershell -c "c:\ffplay -headers ""User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0`r`nCookie: nlqptid=h=676886edeea5bae904b0cf53daec8038.1519940538670&hdnea=exp=1519940598~acl=/*~hmac=C0E019502B060D23AB02BB157FCFFC72404500770A2CE5B00789A84AAEFBD77F&nltid=xxxxxxxxxx&nltdt=0&uid=744826&csid=xxxxxxxxxx; hdntl=exp=1520026938~acl=%2f*~hmac=952689e6de57a2a201ddc1d4c0794962fdc886ea48cf41494a42e787ef923bf9`r`n"" -i ""https://xxxxxxxxxx.net/nlds_vod/xxxxxxxxxx/vod/2018/02/28/cd264e10-dd54-3de2-df90-4c1cac104dcb/v1/stream/cd264e10-dd54-3de2-df90-4c1cac104dcb_1_4500_pc.mp4.m3u8"""

ffplay says

An input file must be specified

But the input file is specified after -i parameter.

What is wrong with this command?

Note that the implementation is a part of the existing batch script, therefore it is required to use a syntax working inside of batch file.

Solution

  • When calling from cmd.exe, things get tricky; here's a simplified example:

    powershell -c " ffmpeg.exe -h \""User-Agent: usra`r`nCookies: parm=a=1&b=2; session=1\"" "

    In short: From cmd.exe / batch files, to pass what PowerShell's CLI should see as a double-quoted argument inside an overall "..." string passed to -Command (-c)[1] (which is powershell.exe's default parameter):

    • If possible, use \"...\", the preferred form of escaping, because it works in both PowerShell editions and also aligns with how most CLIs expect " to be escaped; however, it doesn't work in all cases when calling from cmd.exe, such as the case at hand.
      By contrast, \"...\" is always safe to use from no-shell contexts, such as Task Scheduler and the Windows Run dialog (WinKey-R).

    • To also handle all edge cases of calling the PowerShell CLI from cmd.exe / batch files - namely when the string to be quoted contains cmd.exe metacharacters such as | or & (as in your case) - use the following in lieu of \"...\", again inside overall "..." quoting:

      • When calling Windows PowerShell (powershell.exe): use \""...\"" (sic), as shown above.

        • Minor caveat: If whitespace normalization (runs of two or more spaces becoming a single space each) is a concern (it rarely is), use "^""..."^"" (sic) and, in for /f loops, " ^^^"\"...\"" (sic) - see the bottom section of this answer.

      • When calling PowerShell (Core) 7 (pwsh.exe): use ""..."", which is fully robust (prevents both misinterpretation by cmd.exe and whitespace normalization).

      • As an aside: As a general requirement, % chars. must be escaped as %% to prevent them from being interpreted as part of a cmd.exe variable reference in batch files. From the command prompt, things are unfortunately more complicated.

    Background information:

    • The command line for PowerShell is best passed as a single, "..."-enclosed string, via parameter -c (short for -Command, which in Windows PowerShell is the default, but that has changed in PowerShell (Core) v6+, which now defaults to -File).

    • Since the PowerShell CLI strips unescaped " characters during command-line parsing, before interpreting the result as PowerShell code, any " instances to be retained as part of the command to ultimately execute must be escaped (note that PowerShell-internally `" is used to escape a "; alternatively, in the context of "..." strings only, "" may be used).

    • \""...\"" (Windows PowerShell) and ""..."" (PowerShell (Core) v6+) inside an overall "..." -c argument ensure that cmd.exe itself interprets ... as being inside a double-quoted string, which is what makes these escaping forms robust.

    • If \"...\" were used inside "..." from cmd.exe (which only recognizes "" as an escaped "), it would in effect see ... as being outside a double-quoted string, which would cause values that contain cmd.exe metacharacters such as & and | to break the command. Compare the following invocations from cmd.exe:

      # OK - prints verbatim: The king of "Rock  &  Roll"
C:\>powershell.exe -c " 'The king of \""Rock  &  Roll\""' "
C:\>pwsh.exe -c " 'The king of ""Rock  &  Roll""' "

# !! BROKEN - cmd.exe interprets "&" as a metachar.
C:\>powershell.exe -c " 'The king of \"Rock  &  Roll\"' "
C:\>pwsh.exe -c " 'The king of \"Rock & Roll\"' "
      • Note: You could fix the use of \" by individually ^-escaping such cmd.exe characters (e.g., ^&), but that requires you to analyze the string carefully and determine which specific parts cmd.exe sees as unquoted. If you make a mistake, the ^ will be retained as a literal part of the string. The advantage of the \"" / "" approach is that you needn't worry about such pitfalls.

    As an aside: 'this '"is "'text' does not create a single string the way it does in Bash; in PowerShell:

    • As a stand-alone expression, it causes a syntax error (try executing it by itself); you'd have to use ('this ' + "is " + 'text') or use a single quoted string ('this is text').

    • As an argument passed to a command (program), it is interpreted as 3 distinct arguments - see this answer for an explanation of this - surprising - behavior.

    [1] Using overall "..." quoting around what then becomes a single -Command argument is conceptually preferable, but not strictly necessary. If overall "..." quoting is not used, \"-escaping of " characters that are to be treated as part of the command is the only supported form - see this answer.