Adding retry to shell script

Question

My goal is to increase the number of retries for this fastq validation which at times fails due to network issues, although there is data for the given NCBI SRA id, but it fails. So I need to increase the retry at least 5 times before aborting.

How do I do the same?

set -x
PS4='[\\d \\t] '

# Check parameter for error
check=0
# Print fastq-dump executable path
echo \$(which fastq-dump)

# Loop through all parameters to check validity
for file in \$@;
do
    cp \${file} .
    # Extract filename for sampleID
    file_basename=\$(basename \${file})
    id=\${file_basename%".id"}
    # Start validation
    echo "Checking \${id}..."
    # Download start of fastq
    fastq-dump $(get_ngc()) -X 1 -Z --split-spot \${id} > \${id}.test.fastq 2> \${id}.test.log
    # Get number of lines downloaded to valildate for error
    numLines=\$(cat \${id}.test.fastq | wc -l)
    if [ \$numLines -gt 0 ]; then
        echo "\${id} has data... OK"
    else
        echo "\${id} does not have data... ERROR"
        check=1
    fi
done
# Exit with error if some fastqs not accessible
if [ \$check -gt 0 ]; then
    echo "ERROR: One or more samples have inaccessible fastqs.. exiting"
    exit 1
fi

Answer 1

This is a tentative answer. Hopefully at the very least it can help you ask a more well-formed question going forward.

You mention in a comment that the backslashes work for you in a "cwl file" but you haven't explained what a "cwl file" is. If you are referring to Common Workflow Language then we'd need to know more about the YAML structure around the script; there are certainly YAML scalar formatting options where backslashes are not necessary.

Without more details, let's just say that for a "shell script" these backslashes are wrecking the functionality, and so I have removed them.

It's also not clear which error condition exactly you want to trap. Presumably fastq-dump is the source of these errors. If it is at all compenently written, you should be able to say simply

if fastq-dump --options arguments; then ...

but I stuck to your clumsier approach of counting the number of output lines because I'm not familiar with this tool and you haven't provided a link to its documentation.

#!/bin/sh
# ^ explicitly name which shell you are using
# See also https://en.wikipedia.org/wiki/Shebang_(Unix)

set -x
PS4='[\\d \\t] '

check=0
# Avoid useless use of echo
# https://www.iki.fi/era/unix/award.html#echo
# Prefer POSIX "command -v" over nonstandard "which"
command -v fastq-dump

# Quote all file names
# https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable
for file in "$@"; do
    cp "$file" .
    # Basename knows how to trim extension
    id=$(basename "$file" .id)
    # Write diagnostics to stderr
    echo "$0: Checking $id..." >&2
    # Truncate log so we can append in a loop; see below
    : > "$id".test.log
    # Loop until success, or retries exhausted
    for retry in 1 2 3 4 5; do
        # Append rather than overwrite error log, in case we retry
        fastq-dump $(get_ngc()) -X 1 -Z --split-spot "$id" > "$id".test.fastq 2>> "$id".test.log
        # Avoid useless cat
        # https://stackoverflow.com/questions/11710552/useless-use-of-cat 
        numLines=$(wc -l < "$id".test.fastq)
        if [ $numLines -gt 0 ]; then
            echo "$0: $id has data... OK" >&2
            break
        else
            echo "$0: $id does not have data... ERROR" >&2
            case $retry in
             5) echo "$0: $id: aborting, after 5 attempts" >&2
                check=1;;
             *) # Sleep before retry
                sleep 5;;
            esac
        fi
    done
done
# Exit with error if some fastqs not accessible
if [ $check -gt 0 ]; then
    echo "$0: ERROR: One or more samples have inaccessible fastqs.. exiting" >&2
    exit 1
fi

Many of the beginner errors here would be detected and often even fixed by https://shellcheck.net/; probably run your scripts through this tool before asking here, to avoid distracting answerers. Obviously, sometimes the fix will also solve the problem you wanted to ask about.

$(get_ngc()) still looks like a syntax error, but I'm guessing it's part of the CWL...?