Search code examples
node.jsasynchronousbatching

NodeJS batching async.parallel

I have

async.parallel(tasksGetContentFromGitHub, function(err, res) {
    // all request over, do something
}

the problem is that I might have a large number of tasks, and each of them is sending a request to GitHub.

Since I am a nice citizen, I don't want to send 1000+ queries at once at GitHub, therefore I would like to batch those requests 10 at the time, and then execute my inner code.

Is there an easy way to do that?

Solution

  • You can try async.parallelLimit:

    async.parallelLimit(tasksGetContentFromGitHub, 10, function(err, res) {
    // all request over, do something
}

    Hope that it can help!