How to get output of k8s job in terraform

Question

I am using kubernetes provider in terraform, and I run a kubernetes job on an EKS cluster on AWS. I want to obtain some pieces of output from the job and store them in my terraform variables (or AWS parameter store) so I can use them in my terraform files in other places.
I am not sure how to do it.
Can the job write this information in a file and put it back into the config map that it received? Or is there a way in terraform to parse the log (the stdout) of the k8s job?

Here is, more or less, my terraform file:

resource "kubernetes_job" "my_job" {

  metadata {
    name = "my_job"
    namespace = var.namespace
  }
  spec {
    template {
      metadata {}
      spec {
       volume {
                name = local.config_name
                config_map {
                            name = kubernetes_config_map.my_job.metadata[0].name
                            default_mode = "0777"
                           }
                }
             container {
                    name        = "my_container"
                    image       = "123456789012.dkr.ecr.us-east-1.amazonaws.com/my_container:6.4.1.37"
                    command     = ["/bin/bash","/opt/utilities/run_pre_db_script.sh"]
                    working_dir = "/opt/utilities"
                    volume_mount {

                                name =  local.config_name
                                mount_path = "/opt/my_container_mount"
                              }
             }
        restart_policy = "Never"
      }
    }
  }
  timeouts {
    create="5m"
  }
  wait_for_completion = true
}

Answer 1

Terraform can't parse the logs the job produces, and changing the configmap values is ill-advised as it will create a difference between your configuration and the state in production.

The best solution I can think of for achieving your task is having the job produce dynamically generated tarraform files/terraform parameter files, commit them to the store where the rest of your terraform files are stored (a git repository, for example).

Once you achieve that you can have the next terraform invocation retrieve the results of the job and incorporate it into the next execution.