Search code examples
daskdask-distributeddask-delayed

Add/Enable timestamp to Dask scheduler/workers console logs

Is there a way to add/enable timestamp to Dask scheduler/workers console logs.?

dask: 0.15.0-py35_0
distributed: 1.17.1-py35_0

If I use the above versions this is not enabled -

Scheduler - 

distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO -   Scheduler at: tcp://192.168.200.23:8600
distributed.scheduler - INFO -       bokeh at:       192.168.200.23:8620
distributed.scheduler - INFO -        http at:       192.168.200.23:8610
distributed.scheduler - INFO - Local Directory: /jenkins_VegaFarm_edi-vf-3-4/workspace/AutoBATS/tmp/5896310/cbecb324-9b7c-46af-8ed7-5075aab3f225
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Register tcp://192.168.200.23:33876
distributed.scheduler - INFO - Register tcp://192.168.200.23:43544
distributed.scheduler - INFO - Register tcp://192.168.200.23:43675
distributed.scheduler - INFO - Register tcp://192.168.200.23:39567
distributed.scheduler - INFO - Register tcp://192.168.200.23:33450
distributed.scheduler - INFO - Register tcp://192.168.200.23:42608
distributed.scheduler - INFO - Register tcp://192.168.200.23:36773
distributed.scheduler - INFO - Register tcp://192.168.200.23:43157
distributed.scheduler - INFO - Starting worker compute stream, tcp:/

Workers - 

distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:33621'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:42826'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:37509'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:36526'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:46298'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:36025'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:46421'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:36029'
distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.200.23:41999'
distributed.worker - INFO -       Start worker at: tcp://192.168.200.23:33876
distributed.worker - INFO -              nanny at:       192.168.200.23:44329
distributed.worker - INFO -               http at:       192.168.200.23:34181
distributed.worker - INFO -              bokeh at:       192.168.200.23:8789
distributed.worker - INFO - Waiting to connect to:  tcp://192.168.200.23:8600
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -       Start worker at: tcp://192.168.200.23:33450
distributed.worker - INFO -              nanny at:       192.168.200.23:39203

Is there any way I can prefix timestamp to these logs.? This will help in debugging some internal crashes.

Kind regards, Jacob.

Solution

  • You can provide a log formatting string in your ~/.dask/config.yaml file. Here is the current default that results in your current logs:

    distributed:
  admin:
    log-format: '%(name)s - %(levelname)s - %(message)s'

    See logging module documentation for more formatting options