Search code examples
linuxrenamescp

rename Files of past two days with the there file creation time as suffix and scp the same

I am stuck with commands in Linux where logic is below. Getting the list of files with Yesterday and they-before-yesterday with creation date and renamining them and transfer via scp. file name should be like <<<YYYYMMDD_creation_date_of_that_filename>>>_filename.

for e.g.

  1. getting output

ls -lrth --time-style=+"%Y%m%d" | egrep "date -d '1 day ago' '+%Y%m%d'| date -d '2 day ago' '+%Y%m%d'" > tmp.txt

-rw-rw-rw- 1 user user 418K 20211225 log.897.gz
-rw-rw-rw- 1 user user 419K 20211225 log.898.gz
-rw-rw-rw- 1 user user 419K 20211225 log.899.gz

  1. renaming. this will rename with creation time of that same file as suffix

    for f in tmp.txt|awk '{print$7}'; do cp "$f" "$(stat -c %Y "$f" | date +%Y%m%d)_$f"; done

  2. transfer all these file in one go

for i in list_of_file_from_above; do scp $i user@server.com:/target/folder;done

I am stuck in somewhere loop. also every time i have to enter password for scp Please help. "tar" option can also be considered.

Solution

  • It seems like you have two questions that are critical blockers.

    1. How do I filter files by date?
    2. How can I use ssh without passwords?

    For the first question, use find. There is a nice clue here but that doesn't quite get you to the finish line. Note that the upper bound is exclusive, so that is why I set it to the current date rather than yesterday.

    find . -type f -newermt $(date +%Y%m%d --date "2 days ago") \! -newermt $(date +%Y%m%d)

    For the second question, you will want to use SSH keys. The documentation for ssh-keygen has good examples and is easy to read. In brief, you will:

    • Create a public/private key pair;
    • Copy the public key to the server (the destination machine for your ssh command);
    • and specify the location of the private key in the scp command with the -i option.

    Tip: since you want to use the key in an automated application, do not enter a key password. This is a risk if your key is compromised, so be smart about it.