Search code examples
parallel-processinggnu-parallel

How to use gnu_parallel to run an executable from a list of files?


I am currently learning to use the gnu_paralle. I have read https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Working-as-xargs--n1.-Argument-appending

and also a few other resources and had some success learning the syntax.

At this moment, I have a g++ compiled executable which takes in an *argv. I currently run it in series by doing the following in bash:

cd ./ && find ./data | grep '\.data.format$' > datalist.tmp;

I then read this list of files in the following order

while read LINE; do executable $LINE; done <datalist.tmp

Now. From reading gnu_parallel documentations I understood that processing the executable in multiple instances can be paralleled.

Will someone help with the syntax here please?


Solution

  • Thank you. I ended up finding the following solution,

    while read LINE; do echo executable.exe $LINE ; done < data.tmp |paralell -j4