On IBM DSX I find that if i leave a long running python notebook running overnight, the kernel dies around the same time (around midnight UTC).
The jupyter log shows :
[I 2017-07-29 23:37:14.929 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel e827e71b-6492-4dc4-9201-b6ce29c2100c restarted
[D 2017-07-29 23:37:14.950 NotebookApp] Starting kernel: [u'/usr/local/src/bluemix_jupyter_bundle.v54/provision/pyspark_kernel_wrapper.sh', u'/gpfs/fs01/user/sc1c-81b7dbb381fb6a-c4b9ad2fa578/notebook/jupyter-rt/kernel-e827e71b-6492-4dc4-9201-b6ce29c2100c.json', u'spark20master']
[D 2017-07-29 23:37:14.954 NotebookApp] Connecting to: tcp://127.0.0.1:42931
[D 2017-07-29 23:37:17.957 NotebookApp] KernelRestarter:
restart apparently succeeded
Kernel log or Jupyter log shows nothing else before this point.
Is there some policy that is being enforced here to kill kernels? or maybe some scheduled downtime each day? Does anybody know why the "KernelRestarter" is kicking in?
The KernelRestarter is not killing anything. It notices that the kernel is gone and starts a new one automatically. DSX has inactivity timeouts, but those would shut down your service altogether rather than kill a kernel. And inactivity timeouts are not tied to a wall clock time. This seems to be a bug in DSX.