Search code examples
bashterminalpipersync

bash - Pipe output from ls to rsync


I'm trying to use rsync to backup folders containing a certain word, using ls, grep and rsync. However rsync does not seem to accept output from grep as input. I've tried the following:

$ ls -d $PWD/** | grep March | rsync -av 'dst'

This does nothing really, even though using just ls -d $PWD/** | grep March produces exactly the list of folders I want to move.

$ ls -d $PWD/** | grep March | xargs -0 | rsync -av 'dst'
$ ls -d $PWD/** | grep March | xargs -0 echo | rsync -av 'dst'
$ ls -d $PWD/** | grep March | xargs -0 rsync -av 'dst'

Many(including dst, here I escape space with \) of the folders contains spaces I thought that might cause problems and found xargs might be of help, but still doesn't move anything.

I have tried the above with sudo, the -avuoption for rsync and -r even though this i included in the -aoption. I usually use the --dry-run option for rsync but I've also tried without. What am I doing wrong? Is it possible to pipe input to rsync like this?

I'm on OSX 10.13.3. GNU bash, version 3.2.57(1)

Thank you.


Solution

  • Using the problematic ls and grep, what you want is likely:

    ls -d "$PWD"/** | grep March | xargs -r -n1 -I'{}' rsync -av '{}' 'dst'
    

    A better method is to use --include-from

    One option is to let bash emulate a named pipe:

    rsync -av0 --include-from=<(find . -path '*March*' -print0) "$PWD"/ /dst/
    

    You can also pipe the find output instead:

    find . -path '*March*' -print0| rsync -av0 --include-from=- "$PWD"/ /dst/
    

    -path is used to find "March" anywhere in the filename. (similar to grep)

    (rsync might have some parameters to do the filtering itself as well, like the --include and --exclude patterns)

    Something like this (untested). See here

    rsync -av --include="*/" --include="*March*" --exclude="*" "$PWD"/ /dst/