Get which promise completed in Promise.race

Question

Context: I need to make a large number of asynchronous calls (think around 300 to 3000 ajax calls) that are parallelizable. However, I do not want to strain the browser or server by calling them all at once. I also didn't want to run them sequentially because of the long time it would take to finish. I settled on running five or so at a time and derived this function to do so:

async function asyncLoop(asyncFns, concurrent = 5) {
    // queue up simultaneous calls 
    let queue = [];
    for (let fn of asyncFns) {
        // fire the async function and add its promise to the queue
        queue.push(fn());
        // if max concurrent, wait for the oldest one to finish
        if (queue.length >= concurrent) {
            await queue.shift();
        }
    }
    // wait for the rest of the calls to finish
    await Promise.all(queue);
};

Where asyncFns is an iterable of (not yet called) asynchronous functions.

Problem: This works, however I found that it's not always true that oldest is the first to be complete. I wanted to modify the function so that it uses Promise.race to wait until the first promise succeeds, then continue from there. Yet, I don't know which promise to remove:

        // if max concurrent, wait for the first one to finish
        if (queue.length >= concurrent) {
            await Promise.race(queue);
            // ??? get race's completed promise
            // queue.splice(queue.indexOf(completed), 1);
        }

I could splice it out of the queue (which is now more of a set I guess) if I just knew the index of which one completed. It doesn't look like I can get the original promise from the derived one that race returns. Suggestions?

Answer 1

The "remove from queue" step should happen by the completed promise itself (using then) instead of relying on the returned promise from Promise.race. It seems this is the only way around it.

async function asyncLoop(asyncFns, concurrent = 5) {
    // queue up simultaneous calls 
    let queue = [];
    let ret = [];
    for (let fn of asyncFns) {
        // fire the async function, add its promise to the queue, and remove
        // it from queue when complete
        const p = fn().then(res => {
            queue.splice(queue.indexOf(p), 1);
            return res;
        });
        queue.push(p);
        ret.push(p);
        // if max concurrent, wait for one to finish
        if (queue.length >= concurrent) {
            await Promise.race(queue);
        }
    }
    // wait for the rest of the calls to finish
    await Promise.all(queue);
};

Npm module: https://github.com/rxaviers/async-pool