Search code examples
linuxbashshellrandomscp

how to pick first machine random out of three in shell?


I have three remote machines (machineA, machineB, machineC) from where I can copy files. If for whatever reason I can't copy from machineA, then I should copy from machineB and if for whatever reason I can't copy from machineB then start copying from machineC.

Below is the single shell command I have and I need to run it on many machines but then it means on all those machines, it will copy from machineA only.

(ssh goldy@machineA 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineA:{} /data/files/') || (ssh goldy@machineB 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineB:{} /data/files/') || (ssh goldy@machineC 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineC:{} /data/files/')

Now is there any way by which I can pick first machine randomly (out of those three) instead of keeping machineA as first always. So pick first machine randomly and keep other two as the backup incase first machine is down? Is this possible to do?

Update:

I have something like this:

  machines=(machineA machineB machineC)
  for machine in $(shuf -e ${machines[@]}); do
      ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ threads }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'
      [ $? -eq 0 ] && break
  done

Solution

  • How about keeping the machine names in a file and using shuf to shuffle them? Then you could create a script like this:

    while read machine; do
        ssh goldy@$machine 'ls -1 /process/snap/20180418/*' | parallel -j5 "scp goldy@$machine:{} /data/files/"
        if [ $? == 0 ]; then
            break
        fi
    done
    

    And the machine file like this:

    machineA
    machineB
    machineC
    

    And call the script like this:

    shuf machines | ./script.sh
    

    Here's a test version that doesn't do anything but shows how the logic works:

    while read machine; do
        echo ssh goldy@$machine 'ls -1 /process/snap/20180418/*'
        echo parallel -j5 "scp goldy@$machine:{} /data/files/"
        executenonexistingcommand
        if [ $? == 0 ]; then
            break
        fi
    done
    

    Addressing your comment to use arrays instead and put everything on a single line:

    shuf -e ${machines[@]} shuffles an array. And to read it back into the array, you need to feed the outpu into readarray. Turning scripts into a single line is just a matter of putting semicolons where we had newlines before.

    machines=( machineA machineB machineC ); for machine in $(shuf -e ${machines[@]}); do ssh goldy@$machine 'ls -1 /process/snap/20180418/*' | parallel -j5 "scp goldy@${machine}:{} /data/files/"; if [ $? == 0 ]; then break; fi; done