Search code examples
bashshellquoting

How to only do quote removal in a shell?


Given a string from an untrusted source, e.g.

MALICIOUS_INPUT="$(awk -F = '/^VERSION=/ {print $2}' /path/to/compromised/etc/os-release | head -n 1)"

is it possible to just apply pure shell quote removal (see Shell Command Language (IEEE Std 1003.1-2017) and Bash manual) to that string i.e. without doing variable expansions, arithmetic expansions, command substitution and similar?

This is needed, for example to parse strings from os-release files without source-ing the files.

Input Expected result
'\"' \"
"\"" "
'$foo${foo}$(pwd)$((1+2))' $foo${foo}$(pwd)$((1+2))
"$foo${foo}$(pwd)$((1+2))" $foo${foo}$(pwd)$((1+2))

Solution

  • Comparing applicability of the preexisting answers on Reading quoted/escaped arguments correctly from a string to this question:

    parse_with_xargs() {
      xargs printf '%s\0' <<<"$*"
    }
    
    parse_with_python() {
      python -c '
    import shlex, sys
    for item in shlex.split(sys.stdin.read()):
        sys.stdout.write(item + "\0")
    ' <<<"$*"
    }
    
    readarray -t example_lines <<'EOF'
    '\"'
    "\""
    '$foo${foo}$(pwd)$((1+2))'
    "$foo${foo}$(pwd)$((1+2))"  
    EOF
    
    for line in "${example_lines[@]}"; do
      printf 'Input line:         %s\n' "$line"
      printf 'Parsed with xargs:  '; parse_with_xargs "$line" 2>&1; echo
      printf 'Parsed with python: '; parse_with_python "$line" 2>&1; echo
      echo
    done
    

    Output:

    Input line:         '\"'
    Parsed with xargs:  \"
    Parsed with python: \"
    
    Input line:         "\""
    Parsed with xargs:  xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option
    
    Parsed with python: "
    
    Input line:         '$foo${foo}$(pwd)$((1+2))'
    Parsed with xargs:  $foo${foo}$(pwd)$((1+2))
    Parsed with python: $foo${foo}$(pwd)$((1+2))
    
    Input line:         "$foo${foo}$(pwd)$((1+2))"  
    Parsed with xargs:  $foo${foo}$(pwd)$((1+2))
    Parsed with python: $foo${foo}$(pwd)$((1+2))