Search code examples
kuberneteskubectlconfigmapkustomize

Kustomize: how to reference a value from a ConfigMap in another resource/overlay?


I have a couple of overlays (dev, stg, prod) pulling data from multiple bases where each base contains a single service so that each overlay can pick and choose what services it needs. I generate the manifests from the dev/stg/prod directories.

A simplified version of my Kubernetes/Kustomize directory structure looks like this:

├── base
│   ├── ServiceOne
│   │   ├── kustomization.yaml
│   │   └── service_one_config.yaml
│   ├── ServiceTwo
│   │   ├── kustomization.yaml
│   │   └── service_two_config.yaml
│   └── ConfigMap
│       ├── kustomization.yaml
│       └── config_map_constants.yaml
└── overlays
    ├── dev
    │   ├── kustomization.yaml
    │   └── dev_patch.yaml
    ├── stg
    │   ├── kustomization.yaml
    │   └── stg_patch.yaml
    └── prod
        ├── kustomization.yaml
        └── prod_patch.yaml

Under base/ConfigMap, config_map_constants.yaml file contains key/value pairs that are non-secrets:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: myApp
  name: global-config-map
  namespace: myNamespace
data:
  aws_region: "us-west"
  env_id: "1234"

If an overlay just needs a default value, it should reference the key/value pair as is, and if it needs a custom value, I would use a patch to override the value.

kustomization.yaml from base/ConfigMap looks like this and refers to ConfigMap as a resource:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - config_map_constants.yaml

QUESTION: how do I reference "aws_region" in my overlays' yaml files so that I can retrieve the value?

For example, I want to be able to do something like this in base/ServiceOne/service_one_config.yaml:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: myApp
    aws_region: ../ConfigMap/${aws_region} #pseudo syntax
  name: service_one
spec:
  env_id: ../ConfigMap/${env_id} #pseudo syntax

I am able to build the ConfigMap and append it to my services but I am struggling to find how to reference its contents within other resources.

EDIT: Kustomize version: v4.5.2


Solution

  • You can try using https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/replacements/

    For your scenario, if you want to reference the aws-region into your Service labels. You need to create a replacement file.

    replacements/region.yaml

    source:
      kind: ConfigMap
      fieldPath: data.aws-region
    targets:
      - select:
          kind: Service
          name: service_one
        fieldPaths:
          - metadata.labels.aws_region
    
    

    And add it to your kustomization.yaml

    replacements:
      - path: replacements/region.yaml
    

    Kustomize output should be similar to this

    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: myApp
        aws_region: us-west-1
      name: service_one