Search code examples
apache-storm

Configuration of workers in a storm cluster


I have a question about configuration of worker processes.

I have already understood that worker processes run within a worker node (which is a machine). What I would like to know is if all worker processes share the same JVM or if each worker process has its own JVM instance? If the latter is true, so I suppose one should set how much memory each process has to use! So where would this configuration be done?


Solution

    • A worker process executes a subset of a topology, and runs in its own JVM.
    • A worker process belongs to a specific topology and may run one or more executors for one or more components (spouts or bolts) of this topology.
    • An executor is a thread that is spawned by a worker process and runs within the worker’s JVM.
    • An executor may run one or more tasks for the same component (spout or bolt).
    • An executor always has one thread that it uses for all of its tasks, which means that tasks run serially on an executor.

    So it can be said that all the worker processes will run on the same JVM only belonging to a specific worker node.