I'm making some sort of web scraper in Node.js, that takes a picture of a map that appears on a website with PhantomJS.
However, once the page has been opened, a loading message appears where the map should be. Once the map is ready, the message disappears (visibility: hidden
) and the map is shown.
Because of this, I can't call page.render()
until #loader
is hidden
(or I would get a picture of the loading message, not cool).
// ... Open the page
.then(function(content) {
return page.evaluate(function() {
// Wait for #loading to get hidden somehow ...
var clipRect = document.getElementById('map').getBoundingClientRect();
return {
top: clipRect.top,
left: clipRect.left,
width: clipRect.width,
height: clipRect.height
};
});
})
// Render and process the picture ...
I considered using a mutation observer, but couldn't find a way to use it, since I'm in a promise chain and an event listener wouldn't work as I want.
I also tried checking for the visibility
attribute very often until it became hidden, as explained here, but PhantomJS reported through Node's console:
TypeError: null is not an object (evaluating 'child.transform')
Besides, I'd like to avoid that kind of workarounds if possible, because they're very CPU-intensive.
Any ideas on how can I wait for #loader
to get hidden
under these circumstances?
I finally solved this thanks to phantomjs-node
's mantainer, amir20, so all credit to him. As he explains in this issue:
waitFor
expects to return a value. Butevaluate
returns aPromise
. So that's why it is not working. This is not a problem of the module but rather problem withwaitFor
. Since everything is executed asynchronously then you have to wait for the value.
The function in question (created by him) is the following:
function waitUntil(asyncTest) {
return new Promise(function(resolve, reject) {
function wait() {
asyncTest().then(function(value) {
if (value === true) {
resolve();
} else {
setTimeout(wait, 100);
}
}).catch(function(e) {
console.log('Error found. Rejecting.', e);
reject();
});
}
wait();
});
}
Therefore, applied to my specific example, it should be used like this:
waitUtil(function() {
return sitepage.evaluate(function() {
return document.querySelectorAll('#loader').style.visibility == "hidden";
})
}).then(function(){ // #loading is now hidden
return page.evaluate(function() {
var clipRect = document.getElementById('map').getBoundingClientRect();
return {
top: clipRect.top,
left: clipRect.left,
width: clipRect.width,
height: clipRect.height
};
});
})
// Render and process the picture ...