Search code examples
arraysbashvariablesescapingrsync

Bash: Problem with rsync --exclude using variables with spaces?

I have a list of files/folders I wish to exclude from an rsync command, in the variable $rsyncExclude. Some files/folders have spaces and other special characters ($RECYCLE.BIN) for example.

I'm using the suggestion from @chepner in this question about how to use rsync with options as a variable, but I'm still having trouble getting --exclude to work properly with my use case. Folders such as "/System Volume Information/" etc still copy over.

I've tried using different quotes for $rsyncExclude, e.g.

rsyncExclude="'.cache', ... '[$]RECYCLE.BIN/','/System Volume Information/' ... "

and

rsyncExclude=".cache", ... "[$]RECYCLE.BIN/","/System Volume Information/" ...

and no quotes between:

rsyncExclude=".cache, ... [$]RECYCLE.BIN/,/System Volume Information/ ... "

But no luck... What am I doing wrong?

Here's my script:

#!/bin/bash

rsyncExclude=".cache/,/dev/,/proc/,/sys/,/tmp/,/mnt/,/media/,/lost+found,/Adobe/,[$]RECYCLE.BIN/,/System Volume Information/,pagefile.sys,/temp/"

From="/home/user/bash/backup/testA/"
To="/home/user/bash/backup/testA backup/"


# variables
# thanks to idea by @chepner (https://stackoverflow.com/questions/19219774)
options=(--dry-run --verbose --archive --human-readable --progress --stats --delete-after --itemize-changes --exclude={"${rsyncExclude}"} --delete-excluded)
# rsync
rsync "${options[@]}" "${From}" "${To}"

Update:

Using an array, doesn't seem to work either:

#!/bin/bash

rsyncExclude=(".cache/", "/dev/", "/proc/", "/sys/", "/tmp/", "/mnt/", "/media/", "/lost+found", "/Adobe/", "[$]RECYCLE.BIN/", "/System Volume Information/", "pagefile.sys", "/temp/")

From="/home/user/bash/backup/testA/"
To="/home/user/bash/backup/testA backup/"


# variables
# thanks to idea by @chepner (https://stackoverflow.com/questions/19219774)
options=(--dry-run --verbose --archive --human-readable --progress --stats --delete-after --itemize-changes --exclude={"${rsyncExclude[@]}"} --delete-excluded)
# rsync
rsync "${options[@]}" "${From}" "${To}"

I get the following output:

rsync: link_stat "/dev/," failed: No such file or directory (2)
rsync: link_stat "/proc/," failed: No such file or directory (2)
rsync: link_stat "/sys/," failed: No such file or directory (2)
rsync: link_stat "/tmp/," failed: No such file or directory (2)
rsync: link_stat "/mnt/," failed: No such file or directory (2)
rsync: link_stat "/media/," failed: No such file or directory (2)
rsync: link_stat "/lost+found," failed: No such file or directory (2)
rsync: change_dir "/Adobe" failed: No such file or directory (2)
rsync: change_dir "/home/user/bash/backup//[$]RECYCLE.BIN" failed: No such file or directory (2)
rsync: change_dir "/System Volume Information" failed: No such file or directory (2)
rsync: link_stat "/home/user/bash/backup/pagefile.sys," failed: No such file or directory (2)
rsync: change_dir "/temp" failed: No such file or directory (2)
skipping directory .
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1196) [sender=3.1.2]

Trouble is, those files/paths DO exist in the test directory---i.e. $RECYCLE.BIN, System Volume Information, .cache, pagefile.sys, etc...

Solution

  • Answer for this script, thanks to @Gordon Davisson

    #!/bin/bash

rsyncExclude=(".cache/" "/dev/" "/proc/" "/sys/" "/tmp/" "/mnt/" "/media/" "/lost+found" "/Adobe/" "[$]RECYCLE.BIN/" "/System Volume Information/" "pagefile.sys" "/temp/")

From="./testA a/"
To="./testA-backup wa/"


# using variables
# thanks to idea by @chepner (https://stackoverflow.com/questions/19219774)
# and solution to expanding the --exclude array, thanks to @ Gordon Davisson https://stackoverflow.com/questions/50710476
options=(-vahP --dry-run --stats "${rsyncExclude[@]/#/--exclude=}")
# rsync
rsync "${options[@]}" "${From}" "${To}"

    "${rsyncExclude[@]/#/--exclude=}" does three things; first, the [@] makes it expand to all of the elements of the array; then the /#/--exclude= "replaces" the beginning of each element with "--exclude=" (essentially, it prepends "--exclude=" to each element)