I have a script that is getting me the results I want. I want to improve the performance of the script.
My script takes argument from file file1.txt.
The contents are below:
table1
table2
table3
and so on
Now when I use the while statement like below the script runs in sequential order.
while statement is below:
while IFS=',' read -r a; do import.sh "$a"; done < file1.txt
Now when I use the xargs max-procs utility in bash then the scripts run in parallel based on no of max-procs.
xargs statement is below:
xargs --max-procs 10 -n 1 sh import.sh < file1.txt
Now I have another script
This script takes arguments from file file2.txt.
The contents are below:
table1,db1
table2,db2
table3,db3
and so on
when I use the while statement the script performs fine.
while IFS=',' read -r a b; do test.sh "$a" "$b"; done < file2.txt
But when I use teh xargs statement then the script gives me usage error.
xargs statement is below.
xargs --max-procs 10 -n 1 sh test.sh < file2.txt
The error statement is below:
Usage : test.sh input_file
Why is this happening?
How can I rectify this?
Your second script, test.sh, expects two arguments, but xargs is feeding it only one (one word, in this case the complete line). You can fix it by first converting commas , to newlines (with a simple sed script) and then passing two arguments (now two lines) per call to test.sh (with -n2):
sed s/,/\\n/g file2.txt | xargs --max-procs 10 -n2 sh test.sh
Note that xargs supports a custom delimiter via -d option, and you could use it in case each line in file2.txt were ending with , (but then you should probably strip a newline prefixed to each first field).