Search code examples
bashunixscpxargs

Scp multiple files created in last one day from server to local fails


I want to get multiple files that were generated in last 1 day, from server to local. I'm using the following command which says "No such file or directory".

find username@server.xyz.com:/path-from-which-files-need-to-be-copied/ -type f -ctime -1 | xargs -ILIST scp LIST /Users/abcUser/Documents/test/

Error find: username@server.xyz.com:/path-from-which-files-need-to-be-copied/: No such file or directory

PS: I can access this location and scp from this location for single files with filenames.


Solution

  • find can find only local files. So run find on remote server, something along:

    ssh username@server.xyz.com find /path-from-which-files-need-to-be-copied/ -type f -ctime -1 |
    xargs -ILIST scp username@server.xyz.com:LIST /Users/abcUser/Documents/test/
    

    Note that xargs by default parses \ and quotes in it's own way. The best way to pass the result of find is to use zero terminated stream:

    ssh username@server.xyz.com find /path-from-which-files-need-to-be-copied/ -type f -ctime -1 -print0 |
    xargs -0 -ILIST scp username@server.xyz.com:LIST /Users/abcUser/Documents/test/   
    

    But xargs will invoke separate scp session for each file, which will be very slow. So optimize it by running a single scp for all files, I think you could do it something like this:

    ssh username@server.xyz.com find /path-from-which-files-need-to-be-copied/ -type f -ctime -1 -printf 'username@server.xyz.com:%p\\0' |
    xargs -0 sh -c 'scp "$@" "$0"' /Users/abcUser/Documents/test/