Search code examples
javascriptnode.jspromisephantomjsmutation-observers

Wait in a promise chain for a mutation in a DOM element


I'm making some sort of web scraper in Node.js, that takes a picture of a map that appears on a website with PhantomJS.

However, once the page has been opened, a loading message appears where the map should be. Once the map is ready, the message disappears (visibility: hidden) and the map is shown.

Because of this, I can't call page.render() until #loader is hidden (or I would get a picture of the loading message, not cool).

// ... Open the page

.then(function(content) {
  return page.evaluate(function() {
    // Wait for #loading to get hidden somehow ...
    var clipRect = document.getElementById('map').getBoundingClientRect();
    return {
      top: clipRect.top,
      left: clipRect.left,
      width: clipRect.width,
      height: clipRect.height
    };
  });
})

// Render and process the picture ...

I considered using a mutation observer, but couldn't find a way to use it, since I'm in a promise chain and an event listener wouldn't work as I want.

I also tried checking for the visibility attribute very often until it became hidden, as explained here, but PhantomJS reported through Node's console:

TypeError: null is not an object (evaluating 'child.transform')

Besides, I'd like to avoid that kind of workarounds if possible, because they're very CPU-intensive.

Any ideas on how can I wait for #loader to get hidden under these circumstances?


Solution

  • I finally solved this thanks to phantomjs-node's mantainer, amir20, so all credit to him. As he explains in this issue:

    waitFor expects to return a value. But evaluate returns a Promise. So that's why it is not working. This is not a problem of the module but rather problem with waitFor. Since everything is executed asynchronously then you have to wait for the value.

    The function in question (created by him) is the following:

    function waitUntil(asyncTest) {
        return new Promise(function(resolve, reject) {
            function wait() {
                asyncTest().then(function(value) {
                    if (value === true) {
                        resolve();
                    } else {
                        setTimeout(wait, 100);
                    }
                }).catch(function(e) {
                    console.log('Error found. Rejecting.', e);
                    reject();
                });
            }
            wait();
        });
    }
    

    Therefore, applied to my specific example, it should be used like this:

    waitUtil(function() {
        return sitepage.evaluate(function() {
            return document.querySelectorAll('#loader').style.visibility == "hidden";
        })
    }).then(function(){  // #loading is now hidden
        return page.evaluate(function() {
            var clipRect = document.getElementById('map').getBoundingClientRect();
            return {
                top: clipRect.top,
                left: clipRect.left,
                width: clipRect.width,
                height: clipRect.height
            };
        });
    })
    
    // Render and process the picture ...