Search code examples
mergeyamlyq

YML Deep Merge with YQ


I need to perform what you might call a deep merge between two YML files.

Where there are two matching paths the child nodes should be aggregated, that includes array values. Where there is a path match on a terminal node the value from the second file wins.

For exmaple:

INPUT: a.yml

a:
  a:a
  b:b

b:
  a:a

INPUT: b.yml

a:
  a:1
  c:1

c:
  a:1

REQUIRED OUTPUT:

a:
  a:1
  b:b
  c:1
b:
  a:a
c:
  a:1

The closest I have got to this using yq is:

yq eval-all '. as $item ireduce ({}; . *+d $item )' *.yml

a: a:1 c:1
b: a:a
c: a:1

as you can see a: branch is replaced entirely from the root.


Solution

  • YAML requires at least one whitespace character after the : to make it an object with key and value. In your sample file a.yml containing

    a:
      a:a
      b:b
    

    the value of .a is simply the string a:a b:b. There are no paths .a.a or .a.b.


    With the necessary whitespace added, your attempt works as expected (spelling out the input file arguments because order matters: the latter will overwrite the former).

    yq eval-all '. as $item ireduce ({}; . *+d $item )' a.yml b.yml
    
    a:
      a: 1
      b: b
      c: 1
    b:
      a: a
    c:
      a: 1
    

    Tested with mikefarah/yq v4.35.1