Search code examples
bashbackuprsync

rsync: inclusion of only specific directories does not always work

I am writing a script to create incremental backups with rsync. I have an includes file to only include specific directories of a sub-directory. The rest should be excluded. Everything works as expected with the source directory I am testing with, but it does not when I try to actually backup my files to my external drive.

Here is the script so far:

SOURCE_DIR="$HOME/"
DEST_DIR="/Volumes/Stuff/Backup/"

if [ ! -d "$SOURCE_DIR" ]; then
    echo "\"${SOURCE_DIR}\" does not exist."; exit
elif [ ! -d "$DEST_DIR" ]; then
    echo "\"${DEST_DIR}\" does not exist."; exit
fi

PARENT_DIR="$(cd "$(dirname "$0")"; pwd)"
DIR_NAME="$(date '+%Y%m%dT%H%M%S')"
BACKUP_DIR="$DEST_DIR/$DIR_NAME"
LATEST_LINK="$DEST_DIR/latest"
INCLUDE_FILE="$PARENT_DIR/include.txt"

mkdir -p "$BACKUP_DIR"

rsync -av \
  --delete \
  --prune-empty-dirs \
  --include-from="$INCLUDE_FILE" \
  --link-dest="$LATEST_LINK" \
  "$SOURCE_DIR" \
  "$BACKUP_DIR"

rm -f "$LATEST_LINK"
ln -s "$BACKUP_DIR" "$LATEST_LINK"

echo "Size of current backup:" "$(du -sh "$BACKUP_DIR")"
echo "Size of all backups:" "$(du -sh "$DEST_DIR")"

The file include.txt has the following content:

- .DS_Store
- /Downloads/
+ /Library/
+ /Library/Application Support/***
+ /Library/Preferences/***
- /Library/***

The paths in include.txt are relative to the source directory I assume. Basically I want to back up my home directory, but exclude Downloads and everything in Library except its sub-directories Application Support and Preferences.

I already searched for an answer and as far as I understood it, some had similar problems, but the include file looks alright. And it works with the test directories. Where do I have it wrong?

I expected Downloads and everything in Library except its sub-directories Application Support and Preferences to be excluded, but they are always being included. I changed includes.txt because at first it did not work at all. Now it does work when I backup from $HOME/backup/test-source/ to $HOME/backup/test-dest but not when I actually want to backup from $HOME/ to /Volumes/Stuff/Backup/.

Thanks for your time :)

Solution

  • Problem is the specifications in your "include-from" file.

    There should be no leading / for relative path references.

    Note that the "***" in exlude patterns does not block creation of the parent directory, only the files contained.

    Also, the excludes or includes need to be presented in the order of precedence, which rule overrides subsequent rules, to ensure that those don't get ignored ... because they might fall under the scope of the preceding include/exclude (i.e. + Library/ in your original overrides the -Library/*** ).

    Your file should have the following contents:

    - Downloads/***
- Downloads
+ Library/Application Support/***
+ Library/Preferences/***
- Library/***
+ *
+ .*