Search code examples
jsonstreamuniquejq

Get unique strings from group of strings using JQ

I have strings extracted from jq , from which I just want to get unique values.

"a-b-c-v001"
"a-b-c-v002"
"a-b-c-v001"
"a-b-c-v003"
"a-b-c-v002"

I just need 3 results, unique

"a-b-c-v001"
"a-b-c-v002"
"a-b-c-v003"

I have tried unique & sort did not work - https://jqplay.org/s/xjND6Iv60T

Solution

  • Make sure your jq expression produces an array. For example, if your jq expression (the one producing a stream of strings) is E, then you could modify it to:

    [E] | unique | .[]

    or just:

    [E] | unique[]

    Avoiding a sort

    unique involves a sort, which can easily be avoided if E as above produces a stream of strings:

    INDEX(E;.)[]

    So, in answer to a Q in a comment, a sort-free way to get the count of distinct strings would be:

    def count(s): reduce s as $x (0;.+1);
count( INDEX(E;.)[] )