upgrading to yq v4 from v3 - merging YAML files

Question

I have been using yq v3 to merge yaml files using the following command.

yq m <yaml_file> <yaml_file2> <yaml_file3>

The command works as outlined here

For example, let's say I have the following 3 yaml files.

training-lab-account.yml

account:
  alias: training-lab-account
  orgEmail: [email protected]
  snsSubscriptions:
    - [email protected]
network:
  vpcs:
    - cidr: 10.253.0.0/16
  snsSubscriptions:
    - [email protected]

account.yml

---
account:
  snsSubscriptions:
    - [email protected]
  orgEmail: REQUIRED
  orgAccountProvisioningEnabled: true
  groups:
    - name: SmileAdmins
      policies:
        - AdministratorAccess
    - name: SmileReadOnly
      policies:
        - ReadOnlyAccess

network.yml

---
network:
  snsSubscriptions:
    - [email protected]
  vpcs:
    - vpcName: Main
      amiPipeline: false
      accountVendingMachine: false
      flowlogs: true
      transitGw: false
      maxAzs: 3
      endpoints:
        gateway:
          - name: s3
          - name: dynamodb
        interface:
          - name: ssm

when I execute yq m training-lab-account.yml account.yml network.yml, I get the following output.

account:
  alias: training-lab-account
  orgEmail: [email protected]
  snsSubscriptions:
    - [email protected]
  orgAccountProvisioningEnabled: true
  groups:
    - name: SmileAdmins
      policies:
        - AdministratorAccess
    - name: SmileReadOnly
      policies:
        - ReadOnlyAccess
network:
  vpcs:
    - cidr: 10.253.0.0/16
      vpcName: Main
      amiPipeline: false
      accountVendingMachine: false
      flowlogs: true
      transitGw: false
      maxAzs: 3
      endpoints:
        gateway:
          - name: s3
          - name: dynamodb
        interface:
          - name: ssm
  snsSubscriptions:
    - [email protected]

Basically, account.yml and network.yml are merged into training-lab-account.yml without overwriting the content in training-lab-account.yml.

yq v4 doesn't support the "m" flag to merge anymore. Actually I feel like yq@4 is a brand new piece of software and is completely different from yq@3.

Long story short, I am trying to merge using eval-all as outlined here: yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml f2.yml

Unfortunately, this overwrites what's in f1.yml from f2.yml - which is not how merge behaves in yq@3.

Since yq@3 is deprecated, I need to upgrade to yq@4. I would appreciate any feedback on how to replicate the merge feature from yq@3 in yq@4.

Answer 1

If you give --inplace, it writes the result to the first given file. Just drop that parameter.

---

Edit: To have the values in training-lab-account.yml take precedence over those in the other files, give it as last parameter. To merge the sequence values in vpcs:, use *d to enable deep array merging. Resulting command:

yq eval-all 'select(fileIndex == 0) *d select(fileIndex == 1) *d select(fileIndex == 2)' account.yml network.yml training-lab-account.yml