Automated way to create multiple kubernetes Job manifests

Question

Cron template

kind: CronJob
metadata:
  name: some-example
  namespace: some-example
spec:
  schedule: "* 12 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: some-example
            image: gcr.io/some-example/some-example
            imagePullPolicy: Always
            env:
             - name: REPO_URL
               value: https://example.com/12/some-example

I need to create multiple Job files with different URLs of REPO_URL over 100s save in a file. I am looking for a solution where I can set Job template and get the required key:value from another file.

so far I've tried https://kustomize.io/, https://ballerina.io/, and https://github.com/mikefarah/yq. But I am not able to find a great example to fit the scenario.

Answer 1

That would be pretty trivial with yq and a shell script. Assuming your template is in cronjob.yml, we can write something like this:

let count=0
while read url; do
  yq -y '
    .metadata.name = "some-example-'"$count"'"|
    .spec.jobTemplate.spec.template.spec.containers[0].env[0].value = "'"$url"'"
  ' cronjob.yml
  echo '---'
  let count++
done < list_of_urls.txt | kubectl apply -f-

E.g., if my list_of_urls.txt contains:

https://google.com
https://stackoverflow.com

The above script will produce:

[...]
metadata:
  name: some-example-0
  namespace: some-example
spec:
  [...]
              env:
                - name: REPO_URL
                  value: https://google.com
---
[...]
metadata:
  name: some-example-1
  namespace: some-example
spec:
  [...]
              env:
                - name: REPO_URL
                  value: https://stackoverflow.com

You can drop the | kubectl apply -f- if you just want to see the output instead of actually creating resources.

---

Or for more structured approach, we could use Ansible's k8s module:

- hosts: localhost
  gather_facts: false
  tasks:
    - k8s:
        state: present
        definition:
          apiVersion: batch/v1beta1
          kind: CronJob
          metadata:
            name: "some-example-{{ count }}"
            namespace: some-example
          spec:
            schedule: "* 12 * * *"
            jobTemplate:
              spec:
                template:
                  spec:
                    containers:
                    - name: some-example
                      image: gcr.io/some-example/some-example
                      imagePullPolicy: Always
                      env:
                       - name: REPO_URL
                         value: "{{ item }}"
      loop:
        - https://google.com
        - https://stackoverflow.com
      loop_control:
        index_var: count

Assuming that the above is stored in playbook.yml, running this with ansible-playbook playbook.yml would create the same resources as the earlier shell script.