Search code examples
bashshellcygwin

Unable to pass flags with quotation marks via Bash variables to an executable

I'm trying to create a script in Bash to call an executable with a large set of arguments. To improve readability, I'm grouping different sets of arguments into variables, and passing those variables to the executable.

The issue I'm having is that Bash seems to evaluate the flags improperly when I pass them through a variable to the executable than when I pass them directly. I'd like to be able to pass any type of flag via a variable if I want, even if it requires quotes.

Passing Arguments Directly

I find that if I do this, then the script works and I get the expected output:

$LINT_EXECUTABLE \
    $SYSTEM_INCLUDES \
    $RTE_INCLUDES \
    $SRC_INCLUDES \
    $LINT_CONFIG_INCLUDES \
    $OBJECT_INCLUDES \
    $INDIRECT_FILES \
    -format="*** LINT: %(%f(%l) %)%t %n: %m" \
    src/**/*.c

Output

--- Module:   src\c\error.c (C)
        Including file src/include\error.h (hdr)
*** LINT: src/include\error.h(8) note 9071: defined macro '__ERROR_H' matches a
    pattern reserved to the compiler [MISRA 2012 Rule 21.1, required]
#define __ERROR_H

// ...

Passing Arguments Through Variables

However, if I do this, then I end up with improper output:

OPTIONS="-format=\"*** LINT: %(%f(%l) %)%t %n: %m\""

$LINT_EXECUTABLE \
    $SYSTEM_INCLUDES \
    $RTE_INCLUDES \
    $SRC_INCLUDES \
    $LINT_CONFIG_INCLUDES \
    $OBJECT_INCLUDES \
    $INDIRECT_FILES \
    $OPTIONS \
    src/**/*.c

Output

$ ./lint.sh
PC-lint Plus 1.1 TRIAL for Windows, Copyright Gimpel Software LLC 1985-2018
LICENSED FOR EVALUATION USE ONLY
evaluation license expires in 17 days
"***
LINT:
^

What I've Tried

I've tried substituting the escaped double quotes in the Bash variable with single quotes, but that made no difference. I also temporarily set LINT_EXECUTABLE to echo to print the evaluated set of arguments to the command-line when running $ ./my-script.sh.

In the case of passing flags directly, I see it output ... -format=*** LINT: %(%f(%l) %)%t %n: %m (i.e. no quotes around the -format value) but if I pass the flag via a variable, I see ... -format="*** LINT: %(%f(%l) %)%t %n: %m" ... instead.

Environment

I'm running this on a Windows 64-bit machine using the latest version of bash available in Cygwin:

$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)

Edit

To clarify, OPTIONS will hold more than just the -format flag. Right now I define it as:

OPTIONS="
    +libh(co-arm_TLE9844_AppKit.h) \
    -header(co-arm_TLE9844_AppKit.h) \
    -wlib(4) \
    -wlib(1) \
    +libdir(C:/Keil_v5/ARM/ARMCC/include) \
    -hsfb^3 \
    -format=\"*** LINT: %(%f(%l) %)%t %n: %m\" \
    -width(160,4)"

In response to the suggestion to quote the variable as "$OPTIONS", I tried this but it gave me a different (but still incorrect) output:

$LINT_EXECUTABLE \
    $SYSTEM_INCLUDES \
    $RTE_INCLUDES \
    $SRC_INCLUDES \
    $LINT_CONFIG_INCLUDES \
    $OBJECT_INCLUDES \
    $INDIRECT_FILES \
    "$OPTIONS" \
    src/**/*.c

This is the evaluated arguments that would get passed to the executable doing this:

$ ./lint.sh
-iC:/Keil_v5/UV4/Lint -iC:/Keil_v5/ARM/ARMCC/include -iC:/Keil_v5/ARM/PACK/ARM/CMSIS/5.3.0/CMSIS/Include -iC:/Keil_v5/ARM/PACK/Infineon/TLE984x_DFP/1.1.1/Device/Include -i./RTE/Device/TLE9844-2QX -i./RTE/_TLE9844_AppKit -i./src/include -i./src/include/drivers -i./src/include/utils -i./config/linting -i./Objects ./config/linting/co-ARMCC-5.lnt ./config/linting/std.lnt
    +libh(co-arm_TLE9844_AppKit.h)     -header(co-arm_TLE9844_AppKit.h)     -wlib(4)     -wlib(1)     +libdir(C:/Keil_v5/ARM/ARMCC/include)     -hsfb^3     -format="*** LINT: %(%f(%l) %)%t %n: %m"     -width(160,4) src/c/error.c src/c/main.c ... (other files)

I've used shellcheck.net as recommended and wrapped all my variables in quotes as suggested. This is the latest iteration of my script:

#!/bin/bash

LINT_EXECUTABLE="pclp64"

SYSTEM_INCLUDES="
    -iC:/Keil_v5/UV4/Lint \
    -iC:/Keil_v5/ARM/ARMCC/include \
    -iC:/Keil_v5/ARM/PACK/ARM/CMSIS/5.3.0/CMSIS/Include \
    -iC:/Keil_v5/ARM/PACK/Infineon/TLE984x_DFP/1.1.1/Device/Include"

RTE_INCLUDES="
    -i./RTE/Device/TLE9844-2QX \
    -i./RTE/_TLE9844_AppKit"

SRC_INCLUDES="
    -i./src/include \
    -i./src/include/drivers \
    -i./src/include/utils"

LINT_CONFIG_INCLUDES="
    -i./config/linting"

OBJECT_INCLUDES="
    -i./Objects"

INDIRECT_FILES="
    ./config/linting/co-ARMCC-5.lnt \
    ./config/linting/std.lnt"

OPTIONS="
    +libh(co-arm_TLE9844_AppKit.h) \
    -header(co-arm_TLE9844_AppKit.h) \
    -wlib(4) \
    -wlib(1) \
    +libdir(C:/Keil_v5/ARM/ARMCC/include) \
    -hsfb^3 \
    -format=\"*** LINT: %(%f(%l) %)%t %n: %m\" \
    -width(160,4)"

$LINT_EXECUTABLE \
    "$SYSTEM_INCLUDES" \
    "$RTE_INCLUDES" \
    "$SRC_INCLUDES" \
    "$LINT_CONFIG_INCLUDES" \
    "$OBJECT_INCLUDES" \
    "$INDIRECT_FILES" \
    "$OPTIONS" \
    src/**/*.c

Shellcheck.net reports no issues with this script now, but the script fails a lot earlier when I try to execute it:

$ ./lint.sh
PC-lint Plus 1.1 TRIAL for Windows, Copyright Gimpel Software LLC 1985-2018
LICENSED FOR EVALUATION USE ONLY
evaluation license expires in 17 days
<command line>  2  error 305: unable to open module '-iC:\Keil_v5\UV4\Lint
    -iC:\Keil_v5\ARM\ARMCC\include
    -iC:\Keil_v5\ARM\PACK\ARM\CMSIS\5.3.0\CMSIS\Include
    -iC:\Keil_v5\ARM\PACK\Infineon\TLE984x_DFP\1.1.1\Device\Include'
Solution

  • The root of the issue seems like you need a way to maintain each argument as a separate word, even if that argument contains spaces/quotes. If the quoting a variable doesn't solve it, storing the args in an array and expanding it will.

    So we create a command dynamically, put each argument in a separate element of an array,

    lint_executable=(
                  pclp64
                )

system_includes=( 
                  '-iC:/Keil_v5/UV4/Lint' 
                  '-iC:/Keil_v5/ARM/ARMCC/include'
                  '-iC:/Keil_v5/ARM/PACK/ARM/CMSIS/5.3.0/CMSIS/Include'
                  '-iC:/Keil_v5/ARM/PACK/Infineon/TLE984x_DFP/1.1.1/Device/Include'
                )

rte_includes=(
                '-i./RTE/Device/TLE9844-2QX'
                '-i./RTE/_TLE9844_AppKit'
             )

src_includes=(
                '-i./src/include'
                '-i./src/include/drivers'
                '-i./src/include/utils'
             )

lint_config_includes=(
                       '-i./config/linting'
                     )   

object_includes=(
                  '-i./Objects'
                )

indirect_files=(
                  './config/linting/co-ARMCC-5.lnt'
                  './config/linting/std.lnt'
               )

options=(
          '+libh(co-arm_TLE9844_AppKit.h)'
          '-header(co-arm_TLE9844_AppKit.h)'
          '-wlib(4)'
          '-wlib(1)'
          '+libdir(C:/Keil_v5/ARM/ARMCC/include)'
          '-hsfb^3'
          '-format="*** LINT: %(%f(%l) %)%t %n: %m"'
          '-width(160,4)'
        )

    and now having packed the array, call the command line with a proper quoted expansion

    "${lint_executable[@]}" \
    "${system_includes[@]}" \
    "${rte_includes[@]}" \
    "${src_includes[@]}" \
    "${lint_config_includes[@]}" \
    "${object_includes[@]}" \
    "${indirect_files[@]}" \
    "${options[@]}" \
    src/**/*.c

    Note that single word elements don't have to be packed in array, for uniformity I've demonstrated it in the above case. Also notice the way of using lowercase names for user-defined variables/arrays. It's more of a recommended shell scripting practice. The idea is since the shell maintains its own set of environment variables which are upper-cased, using lower-case names distinguishes them separately.