Nodejs loop through array of urls in a synchronous way

Question

i've worked with node now for 2 years but cannot solve the following requirements:

• I have an array of ~ 50.000 Parameters
• I need to loop through the array and make a get request to always the same url with the parameter added
• I need to write the result of the url-call back to the array
• It's needed to do this one by one, as i can not call the api with several threads.

I'm sure there is a simple solution for that but everything i tried didn't make the code wait for the get request to return. I know that doing things synchronous in node is not the way we should to things, but in this special situation it is by design that the process shall not go on till the result comes back.

Any hint appreciated

Regards

Answer 1

Use a for loop, use a means of doing the GET request that returns a promise (such as the got() library) and then use await to pause the for loop until your response comes back.

const got = require('got');
const yourArray = [...];

async function run() {
    for (let [index, item] of yourArray.entries()) {
        try {
            let result = await got(item.url);
            // do something with the result
        } catch(e) {
            // either handle the error here or throw to stop further processing
        }
    }
}

run().then(() => {
    console.log("all done");
}).catch(err => {
    console.log(err);
});