Very important, this is powershell specific.
There are many answers to variations of this question whose answer is to use single quotes instead of double quotes
e.g. ssh me@server "invalid_program; echo $?" -> ssh me@server 'invalid_program; echo $?'
This is not possible on windows in powershell with openssh. https://stackoverflow.com/a/75740191/8652920
Given there is a restriction that I have to ssh with double quotes only, is there any way to still force the environment variable expansion to happen on remote?
You can try this on your powershell.
PS C:\...> ssh me@remote "beep; echo $?"
Warning: Permanently added 'remote' (ECDSA) to the list of known hosts.
bash: beep: command not found
True
The stdout should be 127, not True.
% ssh me@remote 'beep; echo $?'
Warning: Permanently added 'remote' (ED25519) to the list of known hosts.
127
bash: beep: command not found
Preface:
This [using single quotes] is not possible on windows in powershell with openssh.
In PowerShell (as opposed to in cmd.exe / batch files / no-shell invocations) using '...' when invoking external programs such as ssh does work even on Windows, because PowerShell translates such arguments into "..."-enclosed (double-quoted) arguments behind the scenes, assuming they contain spaces.[1]
Indeed, using '...' (a single-quoted string) is the simplest solution in your case - see the next section.
$ is a metacharacter in both PowerShell and POSIX-compatible shells such as bash, and in both shells "..." is an expandable i.e. interpolating string.
Therefore, in ssh me@remote "beep; echo $?", it is PowerShell that expands $?, up front,[2] using the then-current value of its definition of $?, which is a Boolean value.
To prevent unwanted up-front interpolation:
Either: Use ssh me@remote "beep; echo `$?", i.e. escape the embedded $ using PowerShell's escape character, ` (the so-called backtick)
Or: Use a verbatim string ('...'), to which no interpolation is applied by design:
ssh me@remote 'beep; echo $?'
[1] That is, PowerShell passes arguments that do not contain spaces unquoted on the process command line constructed behind the scenes. Unfortunately, this can break invocation of batch files (*.cmd, *.bat), because cmd.exe, the batch-file interpreter - inappropriately - parses its command line as if it had been passed from inside a cmd.exe session. Thus, an invocation from PowerShell such as .\foo.cmd 'a&b' breaks and requires awkward workarounds. GitHub issue #15143 is a proposal to prevent such problems, but, regrettably, it died from neglect.
Note that in Unix-like environments this point is moot, because there are no process command lines - argument are invariably passed as an array of verbatim strings.
[2] It follows from the above that if you were to run this command from a POSIX-compatible shell (i.e. from a Unix environment), undesired up-front interpolation would occur too, except that a number would be interpolated, namely the most recently executed command's exit code, which is what $? reflects in those shells.