Search code examples
javascriptnode.jsmultithreadingworker

Number of threads Workers in NodeJS will spawn


I am trying to understand the working of workers in NodeJS. My understanding is everytime we spawn a worker , it will create a new thread with it own Node/V8 instance.

So will the below code spawn 50 threads?

How is it distributed over the cpu cores?

This is the index.js

const { Worker } = require("worker_threads");
var count = 0;

console.log("Start Program");

const runService = () => {
  return new Promise((resolve, reject) => {
    const worker = new Worker("./service.js", {});
    worker.on("message", resolve);
    worker.on("error", reject);
    worker.on("exit", code => {
      if (code != 0) {
        reject(new Error("Worker has stopped"));
      }
    });
  });
};

const run = async () => {
  const result = await runService();
  console.log(count++);
  console.log(result);
};

for (let i = 0; i < 50; i++) {
  run().catch(error => console.log(error));
}

setTimeout(() => console.log("End Program"), 2000);

and this is the service.js file

const { workerData, parentPort } = require("worker_threads");

// You can do any heavy stuff here, in a synchronous way
// without blocking the "main thread"
const sleep = () => {
  return new Promise(resolve => setTimeout(() => resolve, 500));
};
let cnt = 0;
for (let i = 0; i < 10e8; i += 1) {
  cnt += 1;
}
parentPort.postMessage({ data: cnt });

Solution

  • So will the below code spawn 50 threads?

    ....
    
    for (let i = 0; i < 50; i++) {
      run().catch(error => console.log(error));
    }
    

    Yes.

    How is it distributed over the cpu cores?

    The OS will handle this.

    Depending on the OS, there is a feature called processor affinity that allow you to manually set the "affinity" or preference a task has for a CPU core. On many OSes this is just a hint and the OS will override your preference if it needs to. Some real-time OSes treat this as mandatory allowing you more control over the hardware (when writing algorithms for self-driving cars or factory robots you sometimes don't want the OS to take control of your carefully crafted software at random times).

    Some OSes like Linux allow you to set processor affinity with command line commands so you can easily write a shell script or use child_process to fine-tune your threads. At the moment there is no built-in way to manage processor affinity for worker threads. There is a third party module that I'm aware of that does this on Windows and Linux: nodeaffinity but it doesn't work on Max OSX (and other OSes like BSD, Solaris/Illumos etc.).