Search code examples
javascriptnode.jsphantomjsx-ray

x-ray for grabbing html info from many urls with "for" loop are letting objs undefined


I'm trying to grab info from many urls automatic. I have this array with address called arrayDep and I have a "for" looping my array and entering the web sites. After that, I use x-ray to grab the info I want. At moment I'm using console.log to see it, but later I will add them to my database. The problem is I'm receiving undefined objects after aleatory times and sometimes server busy messages, I think is something with the time I try to run x-ray, so I tried to add a timeout, sadly without success :(

Code:

 for (var i = 0; i < arrayDep.length; i++) {
   x.timeout(4000);
   x('http://www.camara.leg.br/internet/deputado/' + arrayDep[i], {
     title: 'a'
   })(function(err, obj) {
     console.log(obj.title);
   })
 };

Solution

  • You can use a promises library. I would suggest you use Kriskowal's Q Promises.

    Here's the link to the github repo: https://github.com/kriskowal/q

    There are tonnes of tutorials on the web regarding Q integrations. Kriskowal also has a 1 hour long video on youtube where he explains Q and its uses.

    I hope this helps.