I have three remote machines (machineA, machineB, machineC) from where I can copy files. If for whatever reason I can't copy from machineA, then I should copy from machineB and if for whatever reason I can't copy from machineB then start copying from machineC.
Below is the single shell command I have and I need to run it on many machines but then it means on all those machines, it will copy from machineA only.
(ssh goldy@machineA 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineA:{} /data/files/') || (ssh goldy@machineB 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineB:{} /data/files/') || (ssh goldy@machineC 'ls -1 /process/snap/20180418/*' | parallel -j5 'scp goldy@machineC:{} /data/files/')
Now is there any way by which I can pick first machine randomly (out of those three) instead of keeping machineA
as first always. So pick first machine randomly and keep other two as the backup incase first machine is down? Is this possible to do?
Update:
I have something like this:
machines=(machineA machineB machineC)
for machine in $(shuf -e ${machines[@]}); do
ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ threads }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'
[ $? -eq 0 ] && break
done
How about keeping the machine names in a file and using shuf to shuffle them? Then you could create a script like this:
while read machine; do
ssh goldy@$machine 'ls -1 /process/snap/20180418/*' | parallel -j5 "scp goldy@$machine:{} /data/files/"
if [ $? == 0 ]; then
break
fi
done
And the machine file like this:
machineA
machineB
machineC
And call the script like this:
shuf machines | ./script.sh
Here's a test version that doesn't do anything but shows how the logic works:
while read machine; do
echo ssh goldy@$machine 'ls -1 /process/snap/20180418/*'
echo parallel -j5 "scp goldy@$machine:{} /data/files/"
executenonexistingcommand
if [ $? == 0 ]; then
break
fi
done
Addressing your comment to use arrays instead and put everything on a single line:
shuf -e ${machines[@]}
shuffles an array. And to read it back into the array, you need to feed the outpu into readarray
. Turning scripts into a single line is just a matter of putting semicolons where we had newlines before.
machines=( machineA machineB machineC ); for machine in $(shuf -e ${machines[@]}); do ssh goldy@$machine 'ls -1 /process/snap/20180418/*' | parallel -j5 "scp goldy@${machine}:{} /data/files/"; if [ $? == 0 ]; then break; fi; done