Search code examples
yamlyq

yq, transform and merge two arrays


I have two files from which I want to merge two arrays, one of which I have to transform first.

I already had a look at yq's documentation and multiple SO posts, e.g. this one.
But unfortunately I get null but have the feeling I'm close but I'm not sure what I'm missing

File 1

kind: Cluster
apiVersion: kind,x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 1234
        hostPort: 31234

File 2

traefik:
 metadata:
   namespace: dev
 ports:
   metrics:
     nodePort: 32555
   web:
     nodePort: 30000

desired output

kind: Cluster
apiVersion: kind,x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 1234
        hostPort: 31234
      - hostPort:  32555
      - hostPort: 30000

Here's my current implementation with yq v4.40.5

.test = ((select(fi == 0) | .nodes[0].extraPortMappings) *d (((((select(fi == 1) | .traefik.ports[] | {.nodePort: .}) as $item ireduce({}; . * $item)) as $uniqueMap | ($uniqueMap | to_entries | .[]) as $item ireduce([]; . + {"hostPort": $item.value.nodePort))) | del(.. | select(tag == "!!seq" and length == 0)))) | select(fi == 0)' ./file1.yaml ./file2.yaml

I currently just assign to .test just to see if I get back anything, but it's always null and I don't get why.
When try to assign to file2 (i.e. changing the last select to select(fi == 1) it assigns it as expected)

I think I'm missunderstanding something about yq because the following rather simple assignment isn't working either

.test = ((select(fi == 0) | .nodes[0].extraPortMappings)) | select(fi == 1)' ./file1.yaml ./file2.yaml

How can I write my merged array to file1.yaml?


Solution

  • Looking at your sample data, all you want is to turn each value in .traefik.ports[].nodePort from the second file into an object with field hostPort, and add that to the array under .nodes[0].extraPortMappings in the first file:

    yq '.nodes[0].extraPortMappings += [
      {"hostPort": load("file2.yaml").traefik.ports[].nodePort}
    ]' file1.yaml
    
    kind: Cluster
    apiVersion: kind,x-k8s.io/v1alpha4
    nodes:
      - role: control-plane
        extraPortMappings:
          - containerPort: 1234
            hostPort: 31234
          - hostPort: 32555
          - hostPort: 30000
    

    Note: Use the -i flag to write the results back into file1.yaml.

    Tested mikefarah/yq version v4.35.1