Search code examples
bashsshrsync

While loop executes only once when using rsync


I've created a bash script to migrate sites and databases from one server to another: Algorithm:

  1. Parse .pgpass file to create individual dumps for all the specified Postgres db's.
  2. Upload said dumps to another server via rsync.
  3. Upload a bunch of folders related to each db to the other server, also via rsync.

Since databases and folders have the same name, the script can predict the location of the folders if it knows the db name. The problem I'm facing is that the loop is only executing once (only the first line of .pgpass is being completed).

This is my script, to be run in the source server:

#!/bin/bash

# Read each line of the input file, parse args separated by semicolon (:)
while IFS=: read host port db user pswd ; do
    # Create the dump. No need to enter the password as we're using .pgpass
    pg_dump -U $user -h $host -f "$db.sql" $db
    # Create a dir in the destination server to copy the files into
    ssh [email protected] mkdir -p webapps/$db/static/media
    # Copy the dump to the destination server
    rsync -azhr $db.sql user@destination:/home/user
    # Copy the website files and folders to the destination server
    rsync -azhr --exclude "*.thumbnails*" webapps/$db/static/media/ [email protected]:/home/user/webapps/$db/static/media
# At this point I expect the script to continue to the next line, but if exits at the first line
done < $1

This is .pgpass, the file to parse:

localhost:*:db_name1:db_user1:db_pass1
localhost:*:db_name3:db_user2:db_pass2
localhost:*:db_name3:db_user3:db_pass3
# Many more...

And this is how I'm calling it:

./my_script.sh .pgpass

At this point everything works. The first dump is created, and it is transferred to the destination server along with the related files and folders. The problem is the script finishes there, and won't parse the other lines of .pgpass. I've commented out all lines related to rsync (so the script only creates the dumps), and it works correctly, executing once for each line in the script. How can I get the script to not exit after executing rsync?

BTW, I'm using key based ssh auth to connect the servers, so the script is completely prompt-less.


Solution

  • Let's ask shellcheck:

    $ shellcheck yourscript
    
    In yourscript line 4:
    while IFS=: read host port db user pswd ; do
    ^-- SC2095: ssh may swallow stdin, preventing this loop from working properly.
    
    In yourscript line 8:
        ssh [email protected] mkdir -p webapps/$db/static/media
        ^-- SC2095: Add < /dev/null to prevent ssh from swallowing stdin.
    

    And there you go.