Search code examples
gnu-parallel

How to fix 'Unable to open [{2}]' error in Gnu Parallel


I want to parallelize an image processing step which uses two programs at the same time. My code works fine for a single image but when I try to parallelize it, it fails.

The two programs I am using are fx and getkey from USGS Integrated Software for Imagers and Spectrometers. I use fx to perform an arithmetic operation on my input image (which is 'f1' in the code below) and writes it to a new file (which is the 'to' parameter). getkey outputs the value of a requested keyword, which is a number in this case.

In the following code, I am subtracting the output of getkey from my input image, f1, and writing the result to a new file, which is defined by the 'to' parameter. This code works as I expect it to:

 fx f1=W1660432760_1_overclocks_average_lwps5.cub to=testing_fx2.cub equation=f1-$(getkey from=W1660432760_1_overclocks_average_lwps5_stats.txt grpname=results keyword=average)

The problem comes when I try to parallelize it. The following code gives an error, saying 'Unable to open [{2}].'

parallel fx f1={1} to={1.}_minus_avg.cub equation=f1-$(getkey from={2} grpname=results keyword=average) ::: $(find *lwps5.cub) ::: $(find *stats.txt)

The result I am expecting is an output image with pixel values that are smaller by the getkey value compared to the input image.


Solution

  • If the two inputs should be combined in all ways:

    parallel fx f1={1} to={1.}_minus_avg.cub 'equation=f1-$(getkey from={2} grpname=results keyword=average)' ::: *lwps5.cub ::: *stats.txt
    

    If the two inputs should be linked:

    parallel fx f1={1} to={1.}_minus_avg.cub 'equation=f1-$(getkey from={2} grpname=results keyword=average)' ::: *lwps5.cub :::+ *stats.txt
    

    If neither of these solve you issue, then make a shell function that takes 2 arguments:

    doit() {
      arg1="$1"
      arg2="$2"
      # Do all your stuff with getkey and fx
    }
    export -f doit
    # all combinations
    parallel doit ::: *lwps5.cub ::: *stats.txt
    # or linked
    parallel doit ::: *lwps5.cub :::+ *stats.txt