read all values from multiple documents within a single file into an array with yq

Question

I use mikefahra's yq 4.40.5 and try to read values from a files which may contain multiple documents into one array.

I want to get all .port and .nodePort fields into one array which is contained in the file kind-config.yaml

kind-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.ui/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 3333
        hostPort: 30333
     # add the ports from the files below to this array

Here's how the files look like (they are helm service files)

File 1, multiple docs

apVersion: 1
kind: Service
spec:
  ports:
    - name: my-port
      port: 8080
      targetPort: 8080
      nodePort: 30808
---
apVersion: 1
kind: Service
spec:
  type: ExternalName
  externalName: my-name
  ports:
    - name: my-external-name
      port: 8080
      targetPort: 8080
      nodePort: 30666

I can get those information like:
yq ea '.nodes[0].extraPortMappings += [load("file1.yaml") | .[].spec.ports[] | select(.nodePort) | {"containerPort": .port, "hostPort": .nodePort}]' ./kind-config.yaml

File 2, one document

apVersion: 1
kind: Service
spec:
  type: ExternalName
  externalName: my-name
  ports:
    - name: my-external-name
      port: 7000
      targetPort: 7000
      nodePort: 30700

the script from above fails here because .[].spec is not an array, if I change it to .spec it works again.

But I would prefer if I could solve both my use cases with only one script.

So, is it possible to get the data I need with one single script for both files with multiple docs and those which only contain one?

P.S. I dont mean that I can process multiple files simultaneously with one script, but that I can use the same script.

EDIT: I think it has something to do with how I use load because if I just read the the values I can use the same script for both files.
yq ea '[.spec.ports[] | select(.nodePort) | {"containerPort": .port, "hostPort": .nodePort}]' file1.yaml

Seems that load will load all the documents within a file into an array

Answer 1

After loading the file, you can build a gate that either selects a single map or iterates over the sequence: select(kind == "map") // .[]. From then on, you can treat both inputs as structurally identical:

filter='.nodes[0].extraPortMappings += [
  load(strenv(file)) | select(kind == "map") // .[] | .spec.ports[]
  | select(.nodePort) | {"containerPort": .port, "hostPort": .nodePort}
]'

file=file1.yaml yq ea "$filter" ./kind-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.ui/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 3333
        hostPort: 30333
      - containerPort: 8080
        hostPort: 30808
      - containerPort: 8080
        hostPort: 30666
# add the ports from the files below to this array

file=file2.yaml yq ea "$filter" ./kind-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.ui/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 3333
        hostPort: 30333
      - containerPort: 7000
        hostPort: 30700
# add the ports from the files below to this array

Note that for this sample, you actually don't need the ea command. If not needed by other parts of your actual script, you could either change it to e, or drop it altogether. Tested with mikefarah/yq v4.40.5.