How to write a fish function that delegates to jq?

I am trying to write a fish function called jq_select_keys that selects a subset of keys from a given JSON.

The jq incantation to do this piece of magic is:

jq -r "with_entries(select([.key] | inside([\"bar\",\"baz\",\"qux\"])))" file.json

Now I'm trying to define a handy function called jq_select_keys that will take the filename and the keys that I am interested in, and spit out the subset. Here is what I came up with:

function jq_select_keys --description 'Selects given keys from json input'
  set key_names (for key in $argv[2..-1]; echo "\\\"$key\\\""; end)
  set key_names_joined (string join "," $key_names)
  set jq_args "\"with_entries(select([.key] | inside([$key_names_joined])))\""
  echo "Command: jq -r $jq_args $argv[1]"
  jq -r $jq_args $argv[1]
end

When I run jq_select_keys foo.json bar baz qux on my fish shell, I get the following output:

Command: jq -r "with_entries(select([.key] | inside([\"bar\",\"baz\",\"qux\"])))" foo.json
with_entries(select([.key] | inside(["bar","baz","qux"])))

Now, the interesting bit is that I can copy paste the output of the echo statement, and it runs as expected. But the output I get is just the query string that I passed to jq.

I am new to shell programming, so I might have messed up my quotes. But other than that, I have no clue how to get this thing to work!

Solution

I am new to shell programming, so I might have messed up my quotes.

That's basically it.

It appears you've added one layer of quotes too many.

In fish (and most other shells, including e.g. bash and zsh), quotes are only special to the shell when used literally.

That means when a variable contains "some string", then echo $variable will print "some string" - which means that the echo command received the string with the quotes (note: bash would also apply word-splitting here, so it would actually receive "some and string", even though you might assume it is quoted).

I would suggest you go through your function one by one and simply echo the variables. Then imagine you passed that exact string to jq - would it work?

E.g. the $key_names bit:

set key_names (for key in $argv[2..-1]; echo "\\\"$key\\\""; end)

The purpose of this is obviously to escape all the keys (which are the arguments from the second onwards) for use with jq. That means they need to be quoted once. They don't need to be quoted a second time because the shell does not care about non-literal quotes.

So any key should look like "key".

But when we put

printf '%s\n' $key_names

(which will print each key on its own line)

after this, we see

\"bar\"
\"baz\"
\"qux\"

That's two layers of quoting! The quotes themselves, and the backslashes escaping them.

So let's remove one:

set key_names (for key in $argv[2..-1]; echo "\"$key\""; end)

This will result in "bar", "baz" and "qux".

(This can be simplified to set key_names \"$argv[2..-1]\" by using fish's cartesian product)

Now for the next bit:

set jq_args "\"with_entries(select([.key] | inside([$key_names_joined])))\""

Your intention here is to run this as if you had

jq -r "with_entries(select([.key] | inside([\"bar\",\"baz\",\"qux\"])))" foo.json

On the commandline. But when you use variables, you don't need to quote the string inside them again - all the splitting and other expansions have already happened once the value is assigned, and does not happen again when the variable is substituted.

So simply remove the escaped quotes

set jq_args "with_entries(select([.key] | inside([$key_names_joined])))"

and your function should work.

The result is:

function jq_select_keys --description 'Selects given keys from json input'
    set key_names \"$argv[2..-1]\"
    set key_names_joined (string join "," $key_names)
    set jq_args "with_entries(select([.key] | inside([$key_names_joined])))"
    echo "Command: jq -r $jq_args $argv[1]"
    jq -r $jq_args $argv[1]
end