Search code examples
node.jsnightmare

How to perform multiple nightmare function without it hanging up


I'm trying to scrape a webpage with nightmareJS and got stuck.
In my program i pass to the function an array on links which i need to the same data from all of them
The list can be very long (over 60) and if i try to do a 

async.each(Links, function (url, callback) {
    var nightmare = Nightmare(size);
    ...
}

Only the first couple few instances actually return a value , others just hang up and wont load (blank page).When i try to do only three it work perfectly.
How can i fix it? How can i redistribute the work , for example three in parallel and only when all done it will do the next set? One more thought maybe use the same instance and repeat the steps for all the links?

Solution

  • There are two possible solutions:

    • using eachSeries which waits until one operation is done before launching the other one.

    • Or in async.eachpass another argument which limits how many operation are running in the same time.