I'd like to get information about k8s cronjob time. There are so many jobs in my k8s program. So It's hard to count what time there are focused on. I want to distribute my jobs evenly. Is there are way to count cronjob time or sort by time?
I have tried to find a suitable tool that can help with your case. Unfortunately, I did not find anything suitable and easy to use at the same time.
It is possible to use Prometheus + Grafana to monitor CronJobs e.g using this Kubernetes Cron and Batch Job monitoring dashboard.
However, I don't think you will find any useful information in this way, just a dashboard that displays the number of CronJobs in the cluster.
For this reason, I decided to write a Bash script that is able to display the last few CronJobs run in a readable manner.
As described in the Kubernetes CronJob documentation:
A CronJob creates Jobs on a repeating schedule.
To find out how long a specific Job was running, we can check its startTime and completionTime e.g. using the commands below:
# kubectl get job <JOB_NAME> --template '{{.status.startTime}}' # "startTime"
# kubectl get job <JOB_NAME> --template '{{.status.completionTime}}' # "completionTime"
To get the duration of Jobs in seconds, we can convert startTime and completionTime dates to epoch:
# date -d "<SOME_DATE> +%s
And this is the entire Bash script:
NOTE: We need to pass the namespace name as an argument.
#!/bin/bash
# script name: cronjobs_timetable.sh <NAMESPACE>
namespace=$1
for cronjob_name in $(kubectl get cronjobs -n $namespace --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}'); do
echo "===== CRONJOB_NAME: ${cronjob_name} ==========="
printf "%-15s %-15s %-15s %-15s\n" "START_TIME" "COMPLETION_TIME" "DURATION" "JOB_NAME"
for job_name in $(kubectl get jobs -n $namespace --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | grep -w "${cronjob_name}-[0-9]*$"); do
startTime="$(kubectl get job ${job_name} -n $namespace --template '{{.status.startTime}}')"
completionTime="$(kubectl get job ${job_name} -n $namespace --template '{{.status.completionTime}}')"
if [[ "$completionTime" == "<no value>" ]]; then
continue
fi
duration=$[ $(date -d "$completionTime" +%s) - $(date -d "$startTime" +%s) ]
printf "%-15s %-15s %-15s %-15s\n" "$(date -d ${startTime} +%X)" "$(date -d ${completionTime} +%X)" "${duration} s" "$job_name"
done
done
By default, this script only displays the last three Jobs, but it may by modified in the Job configuration using the .spec.successfulJobsHistoryLimit and .spec.failedJobsHistoryLimit fields (for more information see Kubernetes Jobs History Limits)
We can check how it works:
$ ./cronjobs_timetable.sh default
===== CRONJOB_NAME: hello ===========
START_TIME COMPLETION_TIME DURATION JOB_NAME
02:23:00 PM 02:23:12 PM 12 s hello-1616077380
02:24:02 PM 02:24:13 PM 11 s hello-1616077440
02:25:03 PM 02:25:15 PM 12 s hello-1616077500
===== CRONJOB_NAME: hello-2 ===========
START_TIME COMPLETION_TIME DURATION JOB_NAME
02:23:01 PM 02:23:23 PM 22 s hello-2-1616077380
02:24:02 PM 02:24:24 PM 22 s hello-2-1616077440
02:25:03 PM 02:25:25 PM 22 s hello-2-1616077500
===== CRONJOB_NAME: hello-3 ===========
START_TIME COMPLETION_TIME DURATION JOB_NAME
02:23:01 PM 02:23:32 PM 31 s hello-3-1616077380
02:24:02 PM 02:24:34 PM 32 s hello-3-1616077440
02:25:03 PM 02:25:35 PM 32 s hello-3-1616077500
===== CRONJOB_NAME: hello-4 ===========
START_TIME COMPLETION_TIME DURATION JOB_NAME
02:23:01 PM 02:23:44 PM 43 s hello-4-1616077380
02:24:02 PM 02:24:44 PM 42 s hello-4-1616077440
02:25:03 PM 02:25:45 PM 42 s hello-4-1616077500
Additionally, you'll likely want to create exceptions and error handling to make this script work as expected in all cases.