Step function for EKS

Question

I am trying to run a Step function workflow to execute an EKS Run job task on existing EKS cluster. The jobs runs fine, however when next time the step function is invoked it fails since the pervious execution job state exists in Kubernetes.

Is there a way to generate job name in step function with timestamp so that we don't have to delete previous execution and the next job also gets triggered.

{
  "Comment": "A description of my state machine",
  "StartAt": "EKS RunJob",
  "States": {
    "EKS RunJob": {
      "Type": "Task",
      "Resource": "arn:aws:states:::eks:runJob",
      "Parameters": {
        "ClusterName": "MyClusterName",
        "CertificateAuthority": "LS0tLS1CRUd ... UtLS0tLQo=",
        "Endpoint": "https://12341234.yl4.us-east-1.eks.amazonaws.com",
        "Job": {
          "apiVersion": "batch/v1",
          "kind": "Job",
          "metadata": {
            "name": "Job_name_with_timestamp"
          },
          "spec": {
            "template": {
              "metadata": {
                "name": "Job_name_with_timestamp"
              },
              "spec": {
                "containers": [
                  {
                    "name": "my-function-name",
                    "image": "perl",
                    "command": [
                      "perl"
                    ],
                    "args": [
                      "-Mbignum=bpi",
                      "-wle",
                      "print bpi(2000)"
                    ]
                  }
                ],
                "restartPolicy": "Never"
              }
            }
          }
        }
      },
      "End": true
    }
  }
}

Answer 1

You can update the Parameters field in your Task state to generate a dynamic name. Your best best is probably to use something like the example below, which uses dynamic fields in the Context object to generate a name that will be unique and also helpful to tie back to your Step Functions workflow.

This will generate a value for the name field something like arn:aws:states:us-east-1:123456789100:execution:MyStateMachine-r3kkw3vg2:1872b558-8220-3bf2-bf01-56d293dbd4c6-EKS RunJob-0

{
  "Comment": "A description of my state machine",
  "StartAt": "EKS RunJob",
  "States": {
    "EKS RunJob": {
      "Type": "Task",
      "Resource": "arn:aws:states:::eks:runJob",
      "Parameters": {
        "ClusterName": "MyClusterName",
        "CertificateAuthority": "LS0tLS1CRUd ... UtLS0tLQo=",
        "Endpoint": "https://12341234.yl4.us-east-1.eks.amazonaws.com",
        "Job": {
          "apiVersion": "batch/v1",
          "kind": "Job",
          "metadata": {
            "name.$": "States.Format('{}-{}-{}',$$.Execution.Id,$$.State.Name,$$.State.RetryCount)"
          },
          "spec": {
            "template": {
              "metadata": {
                "name": "Job_name_with_timestamp"
              },
              "spec": {
                "containers": [
                  {
                    "name": "my-function-name",
                    "image": "perl",
                    "command": [
                      "perl"
                    ],
                    "args": [
                      "-Mbignum=bpi",
                      "-wle",
                      "print bpi(2000)"
                    ]
                  }
                ],
                "restartPolicy": "Never"
              }
            }
          }
        }
      },
      "End": true
    }
  }
}