Overlay and override two yaml documents with yq

Question

I would like to overlay a yaml document on top another document, updating any arrays to be the overlaid values.

Given the base/source file:

#base.yaml
include:
  - directory: foo/foo1
  - directory: bar/bar1

and the overlay file:

#overlay.yaml
include:
  - directory: foo/foo1
    extra: true
    stuff:
      stuff1: true
      stuff2: true
  - directory: something/else

and the result should look like

#results.yaml
include:
  - directory: foo/foo1
    extra: true
    stuff:
      stuff1: true
      stuff2: true
  - directory: bar/bar1
  - directory: something/else

I think I am close to having it working with yq from this post, but the list elements are not overridden. I instead get duplicates of foo/foo1

yq eval-all '. as $item ireduce ({}; . *+ $item)' base.yaml overlay.yaml produces

#base.yaml
#overlay.yaml
include:
  - directory: foo/foo1
  - directory: bar/bar1
  - directory: foo/foo1
    extra: true
    stuff:
      stuff1: true
      stuff2: true
  - directory: something/else

Removing the + after the * will drop bar/bar1 from the output.

Basically I think I'm operating on the include, not the values of the include keys. I would greatly appreciate any help in getting the overlay working.

Answer 1

Have a look at this example here: https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-arrays-of-objects-together-matching-on-a-key

Basically you will need to:

yq eval-all '
(
  ((.include[] | {.directory: .}) as $item ireduce ({}; . * $item )) as $uniqueMap
  | ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
) as $mergedArray
| select(fi == 0) | .include= $mergedArray
' sample.yml another.yml

Disclosure: I wrote yq