Search code examples
apache-storm

Reload Storm configuration in runtime


I have a configuration parameters which are loaded in storm configuration map, how can these configuration be updated/reloaded every specific period during run-time without restarting the topology ?


Solution

  • I think your best bet would be putting your configuration somewhere other than the Storm configuration map. The topology configuration is distributed to the workers along with the topology when the worker starts. I'm not aware of a way to update it, and updating it would likely require killing the workers, since Storm components often make use of the configuration in the prepare method, which is only run when the worker starts.

    If you put your configuration somewhere else (e.g. on a REST endpoint accessible to your workers, or a distributed key-value store like Redis), you can use tick tuples to reload the configuration periodically. See how to use tick tuples in this answer https://stackoverflow.com/a/51085518/8845188.