I have a large number of .jpeg files that need to be transferred to a remote server. The remote server is a Digital Ocean droplet and I'm on Windows 10. I'm using the Git bash shell.
When I use this code (below) to transfer my files using scp (username and IP address changed for the sake of privacy).
donald@computer MINGW64 ~/Desktop/RETS Media/extra-large-photos
$ scp -r ./*.jpeg [email protected]:/root/srep-3.1.1/dist/assets/extra-large-photos/
It returns this error message:
Segmentation fault (core dumped)
It also creates a file named scp.exe.stackdump. Here are the contents of that file:
Exception: STATUS_STACK_OVERFLOW at rip=001801DC8B6
rax=00000000002FCBF0 rbx=000000018030B570 rcx=00000000FFE03C40
rdx=00000001801FEA40 rsi=000000018030BEC0 rdi=00000000FFFFCDF0
r8 =00000001801FEA42 r9 =0000000000000000 r10=00000001802DFEA0
r11=00000001802E0068 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
rbp=00000000FFFFCCC0 rsp=00000000FFFFCC28
program=C:\Program Files\Git\usr\bin\scp.exe, pid 18872, thread unknown (0x18A4)
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame Function Args
000FFFFCCC0 001801DC8B6 (001800479BE, 00180046A70, 0010040D060, 001801F57E0)
000FFFFCCC0 000004F4BF0 (00180046A70, 0010040D060, 001801F57E0, 00000000000)
000FFFFCCC0 30001000000FF00 (0010040D060, 001801F57E0, 00000000000, 00000000000)
000FFFFCCC0 001800479BE (00000000000, 00000000000, 00000000000, 00000000000)
00000000000 00180045753 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFFFF0 00180045804 (00000000000, 00000000000, 00000000000, 00000000000)
End of stack trace
The error is literally a "Stack Overflow."
This is not a joke. I'm not trying to troll anyone. This is a serious issue. I have to fix this.
So, where do I go from here?
You should not make a star expansion on a "large amount of jpeg files" in MSYS or MINGW systems. This can lead to some serious trouble.
I would suggest creating a tarball of the files using the tar command or a zip file (if you prefer that, but when packing jpeg files together another compression like zip or gzip is unneccessary, will use up CPU and will not result in smaller files as jpeg files are already compressed).
I suggest for documentation purposes you can read the manpage of tar here: https://www.systutorials.com/docs/linux/man/1-tar/
For example to compress a folder called photos with your files in it, do this:
tar -cf photos.tar photos
which will create the file photos.tar. You can transfer this file via scp.
What works unter Linux and should work on MSYS if you are working in bash and not cmd is directly writing to stdout with tar, transfer it via ssh (not scp) and uncompress it directly on the other end again:
tar -cf - photos/ | ssh [email protected] tar -xf - -C ~/
This will uncompress the files to your homedir. If you want to add compression (which I still do not recommend in this case) you can use czf and xzf in both tar command.