Await multiple promises to resolve while returning the value of each promise as soon as it is resolved

Question

The problem

While working with a restful API, I had to make multiple requests to retrieve data around a single search. The problem that I seem to be facing is that as the results are returned from a large database, some promises take FOREVER to resolve.

Current solution

Currently I make all the requests in a loop while adding the promises to an array and then using await Promise.all() to wait for them to be resolved but this makes loading times > 30 seconds at times even when the earliest promise resolved within a few seconds.
I am looking for a way that I can 'Lazy Load' the results in. I do have access to the restful server so any changes in either the front-end or back-end would help however I would prefer the changes to be at the front end.

Edit 1

My bad I didn't put any references to the code that I am currently using. For reference here is what I am currently doing

async function retrieve_data() {
    let request_urls = [list of api endpoints to fetch]
    let promises = []

    for (let url of request_urls)
        promises.push( fetch(url) )

    await promise.all( promises )

    // Do something with the returned results
}

The solution I want

async function retrieve_data() {
    let request_urls = [list of api endpoints to fetch]
    let promises = []

    for (let url of request_urls)
        promises.push( fetch(url) )

    let results = Promise.some_magic_function()
    // Or server side
    res.write(Promise.some_magic_function()) // If results are received in chunks
    

    // Do something with the returned results
}

Edit 2

So while I did find Promise.any() and Promise.all() to fix half of my problems, I seem unable to comprehend on how to have some function which adds the value of fulfilled promises to an array as soon as they are fulfilled. While these solutions do work for my usecase, I just cant help but think that there must be a better solution.

Answer 1

You can handle the individual results as you're populating the array for Promise.all, which you can use to know when they've all finished.

It's tricky without any code in the question to work with, but here's a simple example:

const promises = [1, 2, 3, 4, 5].map(async (num) => {
    const result = await doRequest(num);
    console.log(`Immediate processing for "${result}"`);
    return result; // If you're going to use the contents of the array from `Promise.all`'s fulfillment
});
const allResults = await Promise.all(promises);
console.log(`All processing done, all results:`);
for (const result of allResults) {
    console.log(result);
}

Live Example:

const rndDelay = () => new Promise((resolve) => setTimeout(resolve, Math.round(Math.random() * 1000) + 100));

async function doRequest(num) {
    console.log(`Requesting ${num}`);
    await rndDelay();
    return `Result for ${num}`;
}

async function main() {
    const promises = [1, 2, 3, 4, 5].map(async (num) => {
        const result = await doRequest(num);
        console.log(`Immediate processing for "${result}"`);
        return result; // If you're going to use the contents of the array from `Promise.all`'s fulfillment
    });
    const allResults = await Promise.all(promises);
    console.log(`All processing done, all results:`);
    for (const result of allResults) {
        console.log(result);
    }
}

main()
.catch((error) => console.error(error));

.as-console-wrapper {
    max-height: 100% !important;
}

Notice that the callback we give map (in this example) is an async function, so it returns a promise and we can use await within it.

If you can't use an async wrapper where you're creating the promises, that isn't a problem, you can fall back to .then:

const promises = [1, 2, 3, 4, 5].map((num) => {
    return doRequest(num)
        .then((result) => {
            console.log(`Immediate processing for "${result}"`);
            return result; // If you're going to use the contents of the array from `Promise.all`'s fulfillment
        });
});
const allResults = await Promise.all(promises);
console.log(`All processing done, all results:`);
for (const result of allResults) {
    console.log(result);
}

const rndDelay = () => new Promise((resolve) => setTimeout(resolve, Math.round(Math.random() * 1000) + 100));

async function doRequest(num) {
    console.log(`Requesting ${num}`);
    await rndDelay();
    return `Result for ${num}`;
}

async function main() {
    const promises = [1, 2, 3, 4, 5].map((num) => {
        return doRequest(num)
            .then((result) => {
                console.log(`Immediate processing for "${result}"`);
                return result; // If you're going to use the contents of the array from `Promise.all`'s fulfillment
            });
    });
    const allResults = await Promise.all(promises);
    console.log(`All processing done, all results:`);
    for (const result of allResults) {
        console.log(result);
    }
}

main()
.catch((error) => console.error(error));

.as-console-wrapper {
    max-height: 100% !important;
}