Search code examples
arraysbashio-redirection

Why using array in command cancels output redirection?


I have this (simplified) code:

elems="${array[@]}"
do_cmd "xxx" "for_each \"$elems\" func" ">> $log_file" || return $?

which redirects output to $log_file (as expected).

However, if the code above is simplified as:

do_cmd "xxx" "for_each \"${array[@]}\" func" ">> $log_file" || return $?

then the output is no longer redirected to $log_file (instead it outputs to stdout).

Any ideas why?

Note: internally do_cmd uses eval. The for_each is:

for_each()
{
  local elems=$1
  local func_and_args=$2

  for elem in $elems
  do
    $func_and_args $elem
  done
}

Solution

  • Why The Original Code Broke

    When you put ${array[@]} in a context that can only ever evaluate to a single string, it acts just like ${array[*]}. This is what your "working" code was doing (and why it wouldn't keep looking like it was working if your array values had spaces in their values or were otherwise at all interesting).

    When you put ${array[@]} in a context that evaluates to multiple words (like a command), it generates breaks between words at every boundary between the individual array items.

    Thus, your string starting with for_each was split up into several different arguments, rather than passing the entire array's contents at once; so the redirection was no longer present in do_cmd's $3 at all.


    How To Do This Well

    Starting with some general points:

    • Array contents are data. Data is not parsed as syntax (unless you use eval, and that opens a can of worms that the whole purpose of using arrays in bash is to avoid).
    • Flattening your array into a string (string=${array[@]}) defeats all the benefits that might lead you to use an array in the first place; it behaves exactly like string=${array[*]}, destroying the original item boundaries.
    • To safely inject data into an evaled context, it needs to be escaped with bash 5.0's ${var@Q}, or older versions' printf %q; there's a lot of care that needs to be taken to do that correctly, and it's much easier to analyze code for correctness/safety if one doesn't even try. (To expand a full array of items into a single string with eval-safe escaping for all in bash 5.0+, one uses ${var[*]@Q}).

    Don't do any of that; the primitives you're trying to build can be accomplished without eval, without squashing arrays to strings, without blurring the boundary between data and syntax. To provide a concrete example:

    appending_output_to() {
      local appending_output_to__dest=$1; shift
      "$@" >>"$appending_output_to__dest"
    }
    
    for_each() {
      local -n for_each__source=$1; shift
      local for_each__elem
      for for_each__elem in "${for_each__source[@]}"; do
        "$@" "$for_each__elem"
      done
    }
    
    cmd=( redirecting_to "$log_file" func )
    array=( item1 item2 item3 )
    for_each array "${cmd[@]}"
    

    (The long variable names are to prevent conflicting with names used by the functions being wrapped; even a local is not quite without side effects when a function called from the function declaring the local is trying to reuse the same name).