Java OOM Automatic Heap Dump File Name in K8S

Question

An answer to this question is available at https://stackoverflow.com/a/34077693/1525813 but not from a Kubernetes perspective.

For my k8s app, I specify the JVM options as an environment variable in the k8s YAML (below). This has been giving issues where the pod crashes on container start-up (running Linux) as the command substitution isn't working as expected.

ENV VAR

spec:
  template:
    spec:
      containers:
      - name: myapp
        command: [ '/bin/sh', '-c' ]
        args:
          - exec java $JVM_OPTS -jar myapp.jar
#         - exec startup.sh # For few other apps
        env:
        - name: JVM_OPTS
          value: >-
            -XX:+HeapDumpOnOutOfMemoryError
            -XX:HeapDumpPath=/logs/heapdump/hd_$(date +%F_%H-%M-%S)_pid$$.hprof
            -XX:+CrashOnOutOfMemoryError
            -XX:OnError=/scripts/threaddump.sh
            -XX:ErrorFile=/logs/error/hs_err_$(date +%F_%H-%M-%S)_pid$$.log
            -Xlog:gc*=debug:file=/logs/gc/gc_%t_pid%p.log:time,uptimemillis,pid,tid,level,tags

ERROR LOGS on container startup

Error: Could not find or load main class +%F_%H-%M-%S)_pid$.hprof"
Caused by: java.lang.ClassNotFoundException: +%F_%H-%M-%S)_pid$.hprof"

What am I missing here?

---

SUB-QUESTION

The earlier config was -XX:HeapDumpPath=/logs/heapdump/hd_%t_pid%p.hprof which was not interrupting the container start-up but the heap dump file was being created with the name hd_%t_pid%p.hprof i.e. %t and %p were not being substituted for -XX:HeapDumpPath but were substituted for -Xlog:gc*.

What could be the cause of discrepancy here?

Thanks

Answer 1

See for details https://github.com/kubernetes/kubernetes/issues/112645

I added these values for JVM_OPTS in a shell script and used it to invoke the JAR. Then invoked the script from the k8s YAML args.