Search code examples
bashdatefilecopy

BASH script to copy files based on date, with a catch


Let me explain the tree structure: I have a network directory where several times a day new .txt files are copied by our database. Those files sit on directory based on usernames. On the local disk I have the same structure (directory based on usernames) and need to be updated with the latest .txt files. It's not a sync procedure: I copy the remote file to a local destination and I don't care what happens with it after that, so I don't need to keep it in sync. However I do need to copy ONLY the new files and not those that I already copied. It would look something like:

Remote disk

/mnt/remote/database
+ user1/
+ user2/
+ user3/
+ user4/

Local disk

/var/database
+ user1/
+ user2/
+ user3/
+ user4/

I played with

find /mnt/remote/database/ -type f -mtime +1

and other variants, but it's not working very well.

So, the script i am trying to figure is the following:
1- check /mnt/remote/database recursively for *.txt
2- check the files date to see if they are new (since the last time I checked, maybe maintain a text file with the last time checked on it as a reference?)
3- if the file is new, copy it to the proper destination in /var/database (so /mnt/remote/database/user1/somefile.txt will be copied to /var/database/user1/)

I'll run the script through a cron job.

I'm doing this in C right now, but the IT people are not very good in debugging or writing C and if they need to add or fix something they can handle bash scripts better, which I am not very good at.

Any ideas out there?

thank you!


Solution

  • Edit: The original link is long gone. Redirecting to Wayback machine.

    The solution is here:

    http://web.archive.org/web/20110206061608/http://www.movingtofreedom.org/2007/04/15/bash-shell-script-copy-only-files-modifed-after-specified-date//