Search code examples
javascriptnode.jsconcurrencypromise

Execute promises concurrently with a buffer pool size in Javascript


I have a function with a promises which must be executed n times with different params each time. I want to chain the promises in a way that the script is always working on 3-4 promises at the time.

I made it with promise.all, this executes 3 concurrently and when all the promises resolves it goes on with the next 3.

How to make it work that when one of 3 resolves it starts immediatly with another but always working on max 3 at the time?

for( var i = 0; i < tasks.length; i++){

    if( i > 0 && i%3 == 0 ){

      await Promise.all([
       doTaskFunction(tasks[i]),
        doTaskFunction(tasks[i-1]),
        doTaskFunction(tasks[i-2]),
      ]);
    }

  }

Solution

  • You can achieve this fairly easy using es6-promise-pool:

    const tasks = [
        (param) => new Promise(function(resolve, reject) {
            setTimeout(resolve, 2000, 'foo');
        }),
        () => new Promise(function(resolve, reject) {
            setTimeout(resolve, 2000, 'foo');
        }),
        () => new Promise(function(resolve, reject) {
            setTimeout(resolve, 2000, 'foo');
        }),
        () => Promise.resolve(1),
        () => Promise.resolve(2),
        () => Promise.resolve(3)
     ];
     
     let count = 1;
    
     const promiseProducer = () => {
        while(tasks.length) {
           console.log('processing ' + count++);
           const task = tasks.shift();
           return task(); // optionally you could pass a parameter here
        }
        
        return null;
     }
     
     const pool = new PromisePool(promiseProducer, 3); // concurrent Promises set to 3
     const poolPromise = pool.start();
    
     poolPromise.then(() => { console.log('done!'); })
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/es6-promise-pool.min.js"></script>