Piping large file from Powershell ssh to Linux

I'm trying to use ssh to load a docker image file stored on a Windows 2022 machine to a Linux machine with docker.

From Windows WSL, this command works perfectly:

cat /mnt/c/temp/my_docker_image.tar.gz | ssh -l matt 192.168.1.250 docker image load

However, I don't want to install WSL just for this (I don't need it otherwise); I just want to use PowerShell. I tried this command in PowerShell and it hangs with PowerShell using up large amounts of memory.

Get-Content -Path C:\temp\my_docker_image.tar.gz -Raw -Encoding Byte | ssh -l matt 192.168.1.250 docker image load

Is there a way to get the PowerShell command working? Or a different command other than Get-Content?

Solution

Windows PowerShell (the legacy, ships-with-Windows, Windows-only edition of PowerShell whose latest and last version is 5.1) and PowerShell (Core) 7 up to v7.3.x only "speak text" when communicating with external programs such as ssh, resulting in misinterpretation of binary data.^[1]
- In PowerShell (Core) 7 v7.4+, [byte] data now can be sent to external programs:
```
# PowerShell 7.4+ only.
# Note the use of -AsByteStream instead of -Encoding Byte 
Get-Content C:\temp\my_docker_image.tar.gz -AsByteStream |
  ssh -l matt 192.168.1.250 docker image load
```
- Note the omission of -Raw, which is what makes the solution memory-efficient, because the bytes are relayed one by one, in a streaming fashion.
- That said, it is precisely this byte-by-byte processing that makes this this approach quite slow, so even in 7.4+ you may consider the alternative solution below (while use of -Raw is much faster, it invariably entails reading the entire file in memory, up front, which causes problems with large files, as in your case).

Workaround for Windows PowerShell and PowerShell (Core) 7 versions up to v7.3.x:

Take advantage of the fact that cmd.exe's pipeline is a raw byte conduit (as in POSIX-compatible shells such as bash), so pass an equivalent command line to it:

cmd /c 'type C:\temp\my_docker_image.tar.gz | ssh -l matt 192.168.1.250 docker image load'

As noted, you may choose to continue to use this approach in PowerShell 7.4+, for performance reasons.

For detailed background information, see the bottom section of this answer.

^{[1] Even if you read the input file as binary data (a [byte[]] array), on sending this data through the pipeline it is treated as text in PowerShell versions before 7.4. (each [byte] instance is converted to its decimal string representation, and the instances are separated with newlines).}