Search code examples
bashshellprintfescapingecho

character-escaping double quotes in bash append file of command and output

I work with a variety of diagnostic files (text / jq for JSON / yq for YAML / xpath for XML, lnav for log files) in Bash where I need to dynamically determine meaningful data parsing before reporting the final command and its output.

A bit ago as written-up here after I found out via Echoing the last command run in Bash? that I could use !! to run the previous command and stash its results to a file.

The problem I've encountered is an edge-case that when the ran command uses double-quotes they get omitted when stashed:

$ echo "hello"
hello
$ echo "!!" >> test && !! >> test
echo "echo "hello"" >> test && echo "hello" >> test
$ cat test
echo hello # no double-quotes
hello      # but right answer

I've attempted but failed so far to compensate for this via

The one that's worked best is script, except A) I only need to retain <5% of commands/output ran & B) I haven't figured out to load my Dotfiles (though I assume that answer lies in man script's SCRIPT or SHELL environment variables which I'd investigate further if that's the best bet).

My problem specifically becomes non-cosmetic and interruptive when asking others to repeat queries I've compiled (for example, to monitor resolution steps effectiveness):

# what I would run
$ echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'
a-a

# how I would cache for others to run
$ echo "!!" >> test && !! >> test
echo "echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'" >> test && echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")' >> test

# this command does not work unless !! is within double quotes which partially seems to induce the issue

# what stored to file for others to run
$ cat test
echo '[a-a,b-a]' | jq -rc '.[]|select(.==a-a)'
a-a


# command will error for others because double-quotes dropped
$ echo '[a-a,b-a]' | jq -rc '.[]|select(.==a-a)'
jq: error: a/0 is not defined at <top-level>, line 1:
.[]|select(.==a-a)
jq: error: a/0 is not defined at <top-level>, line 1:
.[]|select(.==a-a)
jq: 2 compile errors

What I expect / hope is

  • there's a flag in either echo or printf that character-escapes the double-quotes when running "!!" or avoids !! needing the double-quotes
  • there's a way to run script (with Dotfiles) to just "run and stash the very last command" (instead of reporting everything and then after manually deleting most of the session)

What I'm not interested in (since I work across Operating Systems and can't guarantee other's installations/environments)

  • Jupyter Notebook has a Bash kernel that I could use and then manually copy/paste into a text file
  • Bash is the most common scripting language across Operating Systems, so while I love Python and ipython would answer very close to my request, my folks aren't guaranteed to have Python installed so it's a no-go.
  • advice to error-handle when double-quotes was involved to manually stash the command+output; I have this now just not in the above write-up
  • needing to write the original query in single-quotes with single-quote escaping to then use "!!" without it dropping double-quotes. That works in a lot of places (and I do it though annoyed) but single-quote-escaping causes issues using Salesforce's CLI.

I reviewed Stack Overflow's recommended questions but did not think they answered:

Solution

  • !! by itself does not swallow any quotes. It happens when the shell is interpreting the substitute literally, which includes resolving quoted content. "!!" doesn't do more than wrapping double quotes around it, which still just contributes to the interpretation by the shell but now it also easily interferes with quotes coming from the previous command.

    A better way would be to retrieve the last command by other means, e.g. history. There are several ways to filter for the last one, here are some examples:

    $ echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'
a-a

$ cmd="$(history -- -2 | head -n1 | cut -c8-)"

    $ echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'
a-a

$ cmd="$(history -w /dev/stdout | tail -n2 | head -n1)"

    $ echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'
a-a

$ cmd="$(HISTTIMEFORMAT="$(echo -e '\r\e[K')" history -- -2 | head -n1)"

    All these examples assign the content of the last command to a shell variable:

    $ printf '%s\n' "$cmd"
echo '["a-a","b-a"]' | jq -rc '.[]|select(.=="a-a")'

    Alternatively, instead of using an assignment cmd="$(…)", you can also redirect it to a file, e.g. … > file, or pipe it into the another processor, e.g. … | sh, etc.