Search code examples
javascriptnode.jsmultithreadinglibuv

Possible to increase UV_THREADPOOL_SIZE on the fly?


I'm experimenting on multithreading on NodeJS and poking around with the code below

const crypto = require('crypto');

const start = Date.now();

crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
  console.log('1:', Date.now() - start);
});

crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
  console.log('2:', Date.now() - start);
});

crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
  console.log('3:', Date.now() - start);
});

crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
  console.log('4:', Date.now() - start);
});

crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
  console.log('5:', Date.now() - start);
});

Correct me if I'm wrong, by default libuv has a threadpool size of 4 as stated here. So by running the code above, each and every crypto.pbkdf2 gets assigned to individual thread and run at the same time, while the fifth will wait for awhile to run. Below is the result

1: 484
2: 490
3: 490
4: 490
5: 963

I can understand the fifth crypto.pbkdf2 double the time due to out of threads to be running parallel, so now I'm exploring increasing the thread pool size. Below is my attempt but the result seems to be identical?

$ set UV_THREADPOOL_SIZE=120 && node threads.js 

2: 487
3: 492
1: 493
4: 493
5: 986

Solution

  • You should verify if you've indeed successfully set the environment variable by simply console.log(process.env.UV_THREADPOOL_SIZE), and if you do, you'll realize that it is undefined.

    The problem is with how you invoke your command, it should be as below instead

    UV_THREADPOOL_SIZE=120 node threads.js