throttle requests in Node.js

Question

I have an array. I can loop over it with the foreach method.

data.forEach(function (result, i) {

     url = data[i].url;

     request(url);

});

The request function is making a http request to the given url. However making all these requests at the same time leads to all sorts of problems.

So I thought I should slow things down by introducing some-sort of timer.

But I have no idea how will be able to combine a forach loop with setTimeOut/setInterval

Please note am doing this on the server (nodejs) rather on the browser.

Thanks for you help.

Answer 1

Instead of setTimeout could have them run in sequence. I assume there's a callback parameter to your request() function.

function makeRequest(arr, i) {
    if (i < arr.length) {
        request(arr[i].url, function() { 
                                i++; 
                                makeRequest(arr, i); 
                            });
    }
}

makeRequest(data, 0);

---

If you need a little more time between requests, then add the setTimeout to the callback.

function makeRequest(arr, i) {
    if (i < arr.length) {
        request(arr[i].url, function() { 
                                i++; 
                                setTimeout(makeRequest, 1000, arr, i); 
                            });
    }
}

makeRequest(data, 0);