I want to parallelize an image processing step which uses two programs at the same time. My code works fine for a single image but when I try to parallelize it, it fails.
The two programs I am using are fx
and getkey
from USGS Integrated Software for Imagers and Spectrometers. I use fx
to perform an arithmetic operation on my input image (which is 'f1' in the code below) and writes it to a new file (which is the 'to' parameter). getkey
outputs the value of a requested keyword, which is a number in this case.
In the following code, I am subtracting the output of getkey
from my input image, f1, and writing the result to a new file, which is defined by the 'to' parameter. This code works as I expect it to:
fx f1=W1660432760_1_overclocks_average_lwps5.cub to=testing_fx2.cub equation=f1-$(getkey from=W1660432760_1_overclocks_average_lwps5_stats.txt grpname=results keyword=average)
The problem comes when I try to parallelize it. The following code gives an error, saying 'Unable to open [{2}].'
parallel fx f1={1} to={1.}_minus_avg.cub equation=f1-$(getkey from={2} grpname=results keyword=average) ::: $(find *lwps5.cub) ::: $(find *stats.txt)
The result I am expecting is an output image with pixel values that are smaller by the getkey
value compared to the input image.
If the two inputs should be combined in all ways:
parallel fx f1={1} to={1.}_minus_avg.cub 'equation=f1-$(getkey from={2} grpname=results keyword=average)' ::: *lwps5.cub ::: *stats.txt
If the two inputs should be linked:
parallel fx f1={1} to={1.}_minus_avg.cub 'equation=f1-$(getkey from={2} grpname=results keyword=average)' ::: *lwps5.cub :::+ *stats.txt
If neither of these solve you issue, then make a shell function that takes 2 arguments:
doit() {
arg1="$1"
arg2="$2"
# Do all your stuff with getkey and fx
}
export -f doit
# all combinations
parallel doit ::: *lwps5.cub ::: *stats.txt
# or linked
parallel doit ::: *lwps5.cub :::+ *stats.txt