Search code examples
ubuntuparallel-processinggnugnu-parallel

gnu parallel: Prefix output with hostname(s)

Is it possible to prefix the output of gnu parallel when I run same command on multiple hosts?

I have 10 worker machines in a worker-pool and any one of them could've picked up the job, I want to find out which worker picked it up by greping log files on all:

parallel --nonall -S host1,host2,host3 grep job_id_123 /var/log/my_log.log

prints something like:

initing job_id_123
doing phase1 job_id_123
doing phase2 job_id_123
wrapping up job_id_123

What I want is to 

host2: initing job_id_123
host2: doing phase1 job_id_123
host3: doing phase2 job_id_123
host1: wrapping up job_id_123

I know I can do this:

parallel --nonall -S host1,host2,host3 "hostname && grep job_id_123 /var/log/my_log.log"

but prefixing is what I was hoping for.

I'm using GNU parallel 20160422 on Ubuntu precise (12.04.5 LTS)

Solution

  • You can use the --tag option to tag each server's output when running with multiple machines. So, in the following example, I am asking the local host (an iMac) and two Raspberry Pis to print their OS name:

    parallel --nonall --tag -S :,pi1,pi2 uname
:   Darwin
pi1 Linux
pi2 Linux