I'm trying to grab info from many urls automatic. I have this array with address called arrayDep and I have a "for" looping my array and entering the web sites. After that, I use x-ray to grab the info I want. At moment I'm using console.log to see it, but later I will add them to my database. The problem is I'm receiving undefined objects after aleatory times and sometimes server busy messages, I think is something with the time I try to run x-ray, so I tried to add a timeout, sadly without success :(
Code:
for (var i = 0; i < arrayDep.length; i++) {
x.timeout(4000);
x('http://www.camara.leg.br/internet/deputado/' + arrayDep[i], {
title: 'a'
})(function(err, obj) {
console.log(obj.title);
})
};
You can use a promises library. I would suggest you use Kriskowal's Q Promises.
Here's the link to the github repo: https://github.com/kriskowal/q
There are tonnes of tutorials on the web regarding Q integrations. Kriskowal also has a 1 hour long video on youtube where he explains Q and its uses.
I hope this helps.