Search code examples
bashmacosrsyncscpfind-util

Bash: Fail to use find -exec

When using scp or rsync I often fail to deal with 'Argument list too long' error. When having to mv or rm, I have no problem to use find and xargs but I fail to understand how to use find and -exec despite all the SE posts on the subject. Consider the following issue...

I tried

$scp /Path/to/B* Me@137.92.4.152:/Path/to/

-bash: /usr/bin/scp: Argument list too long

So I tried

$find . -name "/Path/to/B*" -exec scp "{}" Me@137.92.4.152:/Path/to/ '\;'

find: -exec: no terminating ";" or "+"

so I tried

$find . -name "/Path/to/B*" -exec scp "{}" Me@137.92.4.152:/Path/to/ ';'

find: ./.gnupg: Permission denied
find: ./.subversion/auth: Permission denied

So I tried

$sudo find . -name "/Path/to/B*" -exec scp "{}" Me@137.92.4.152:/Path/to/ ';'

and nothing happen onces I enter my password

I am on Mac OSX version 10.11.3, Terminal version 2.6.1

Solution

  • R. Saban's helpful answer solves your primary problem:

    • -name only accepts a filename pattern, not a path pattern.

    • Alternatively, you could simply use the -path primary instead of the -name primary.

    As for using as few invocations of scp as possible - each of which requires specifying a password by default:

    • As an alternative, consider bypassing the use of scp altogether, as suggested in Eric Renouf's helpful answer.

    • While find's -exec primary allows using terminator + in lieu of ; (which must be passed as ';' or \; to prevent the shell from interpreting ; as a command terminator) for passing as many filenames as will fit on a single command line (a built-in xargs, in a manner of speaking), this is NOT an option here, because use of + requires that placeholder {} come last on the command line, immediately before +.

    • However, since you're on macOS, you can use BSD xarg's nonstandard -J option for placing the placeholder anywhere on the command line, while still passing as many arguments as possible at once (using BSD find's nonstandard -print0 option in combination with xargs's nonstandard -0 option ensures that all filenames are passed as-is, even if they have embedded spaces, for instance):

    find . -path "/Path/to/B*" -print0 | xargs -0 -J {} scp {} Me@137.92.4.152:/Path/to/

    Now you will at most be prompted a few times: one prompt for every batch of arguments, as required to accommodate all arguments while observing the max. command-line length with the fewest number of calls possible.