Search code examples
arraysyamluniqueyq

yq (GO/Mike Farah) uniquify all arrays recursively

This is probably related to another question I posted: yq (GO/Mike Farah) sort all arrays recursively?

Mike Farah's yq provides documentation for making arrays unique but I'm having trouble figuring how to apply that to lists that are nested deeper

Input

classes:
  driver:
    fields:
      - height
      - age
      - age
  vehicle:
    fields:
      - model
      - model
      - color
      - year

Desired output

classes:
  driver:
    fields:
      - age
      - height
  vehicle:
    fields:
      - color
      - model
      - year

Naively trying to globally uniqify

cat to_sort.yaml | yq 'unique'

Error: Only arrays are supported for unique

And if it takes arguments, I don't know what to provide. I don't want to just sort one explicit path, but I did try this:

 cat to_sort.yaml | yq 'unique(.classes.driver.fields)'

Error: Bad expression, please check expression syntax

I have seen some yq examples where one has to do a select operation first, but I don't know what to try in this case.

Solution 
  •  yq e '(... | select(type == "!!seq")) |= unique' input

    Will recursively loop over all the items, and select() those of type !!seq

    Then update (|=) those with unique:

    Result from provided input:

    classes:
  driver:
    fields:
      - height
      - age
  vehicle:
    fields:
      - model
      - color
      - year