I have the following bash script. In the script I use rsync to copy files from a source to a destination. In the first call of rsync I copy all the files and in the second call I double-check the files and if the checksum is valid the copied files are deleted in the source.
#!/bin/bash
set -e
rsync --info=progress2 -r --include='database/session_*.db' --exclude 'database/session*' /local/data/ /import/myNas/data
rsync --info=progress2 -r --include='database/session_*.db' --exclude 'database/session*' --checksum --remove-source-files /local/data/ /import/myNas/data
The problem now is that while rsync is running new files are written to /local/data. I would like that rsync takes a snapshot of the list of files in source (/local/data) when it runs the first time and then only copies these files. In the second run rsync should then also only run on these files from the snapshot (i.e. calculate the checksum and then delete the files). That means the new added files should not be touched.
Is this possible?
Populating a null delimited list of files to synchronize before running rsync with this list:
#!/usr/bin/env bash
##### Settings #####
# Location of the source data files
declare -r SRC='/local/data/'
# Destination of the data files
declare -r DEST='/import/myNas/data/'
##### End of Settings #####
set -o errexit # same as set -e, exit if command fail
declare -- _temp_fileslist
trap 'rm -f "$_temp_fileslist"' EXIT
_temp_fileslist=$(mktemp) && typeset -r _temp_fileslist
# Populate files list as null delimited entries
find "$SRC" \
-path '*/database/session_*.db' \
-and -not -path '*/database/session*' \
-fprinf "$_temp_fileslist" '%P\0'
# --from0 tells rsync to read a null delimited list
# --files-from= tells to read the include list from this file
if rsync --info=progress2 --recursive \
--from0 "--files-from=$_temp_fileslist" -- "$SRC" "$DEST";
then rsync --info=progress2 --recursive \
--from0 "--files-from=$_temp_fileslist" \
--checksum --remove-source-files -- "$SRC" "$DEST"
fi