Search code examples
apache-storm

where can I find the default value for this storm configurable `topology.worker.timeout.secs`


I find the declaration here: https://github.com/apache/storm/blob/3f96c249cbc17ce062491bfbb39d484e241ab168/storm-client/src/jvm/org/apache/storm/Config.java#L1161 but could not find its default value from the yaml (https://github.com/apache/storm/blob/b6e7d0355e0397b8acc566961ed31279338998e1/conf/defaults.yaml). I am trying to tune this parameter in my setup but need to figure out what is a "recommended" value to start with.


Solution

  • The comment on topology.worker.timeout.secs says:

    Topology configurable worker heartbeat timeout before the supervisor tries to restart the worker process. Maximum value constrained by WORKER_MAX_TIMEOUT_SECS. When topology timeout is greater, the following configs are effectively overridden: SUPERVISOR_WORKER_TIMEOUT_SECS, SUPERVISOR_WORKER_START_TIMEOUT_SECS, NIMBUS_TASK_TIMEOUT_SECS and NIMBUS_TASK_LAUNCH_SECS.

    As I can not find a default value for that as well, I had a look into nimbus.java:

    if (mergedConf.containsKey(Config.TOPOLOGY_WORKER_TIMEOUT_SECS)) {
      int workerTimeoutSecs = (Integer) ObjectReader.getInt(mergedConf.get(Config.TOPOLOGY_WORKER_TIMEOUT_SECS));
      int workerMaxTimeoutSecs = (Integer) ObjectReader.getInt(mergedConf.get(Config.WORKER_MAX_TIMEOUT_SECS));
      if (workerTimeoutSecs > workerMaxTimeoutSecs) {
          ret.put(Config.TOPOLOGY_WORKER_TIMEOUT_SECS, workerMaxTimeoutSecs);
          ...
      }
    }
    

    But this key is not contained anywhere, and when I am debugging through nimbus and evaluate that config, I get null, meaning, that this value is never set.

    Looking at WORKER_MAX_TIMEOUT_SECS gives 600 seconds in default.yaml, so I think this might serve as a starting point for you.