I am completely new in using GNU parallel and I need your advice in running the command below using GNU parallel:
/home/admin/Gfinal/decoder/decdr.pl --gh --w14b /data/tmp/KRX12/a.bin |
perl /home/admin/decout/decoder/flow.pl >> /data/tmp/decodedgfile/out_1.txt
I will run this command on a list of files (.bin), so what is the best (fastest) approach to achieve that using GNU parallel noting that the output of the first part of the command (/home/admin/Gfinal/decoder/decdr.pl --gh --w14b) is very large (> 2 GB).
Any help would be appreciated.
Will this work:
parallel /home/admin/Gfinal/decoder/decdr.pl --gh --w14b {} '|' perl /home/admin/decout/decoder/flow.pl >> /data/tmp/decodedgfile/out_1.txt ::: /data/tmp/KRX12/*.bin
(If the output from flow.pl is more than your disk I/O can handle, try parallel --compress).
Or maybe:
parallel /home/admin/Gfinal/decoder/decdr.pl --gh --w14b {} '|' perl /home/admin/decout/decoder/flow.pl '>>' /data/tmp/decodedgfile/out_{#}.txt ::: /data/tmp/KRX12/*.bin
It depends on whether you want a single output file or one per input file.
Also spend an hour walking through the tutorial. Your command line will love you for it. man parallel_tutorial