Search code examples
python-3.xdockerdbusscrapy-splashsplash-js-render

Splash issues (d-bus, QSslSocket, libpng)


I'm trying to use Splash via scrapinghub/splash Docker image and have some alerts coming after the first request (which is to /robots.txt endpoint because I'm using scrapy-splash plugin for scrapy library (with Python 3.6).

[-] "172.17.0.1" - - [18/Jan/2018:00:05:12 +0000] "GET /robots.txt HTTP/1.1" 404 153 "-" "Scrapy/1.5.0 (+https://scrapy.org)"
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
process 1: D-Bus library appears to be incorrectly set up; failed to read machine uuid: UUID file '/etc/machine-id' should contain a hex string of length 32, not length 0, with no other text
See the manual page for dbus-uuidgen to correct this issue.
qt.network.ssl: QSslSocket: cannot resolve SSLv2_client_method
qt.network.ssl: QSslSocket: cannot resolve SSLv2_server_method

And the saddest thing is that it doesn't render this page. Also need to mention that sometimes it renders this page, and it contains redirect in case JS is available.

How to make it work?

UPDATE

When using scrapinghub/splash:3.0 these messages still, but render works. So should I report it as bug in scrapinghub/splash image or these errors can be caused by my environment?

UPDATE

For some reason, even 3.0 doesn't render from now, as well as master (docker image tag). So for every image with latest or master or 3.0 tag, after asking (form on index endpoint) for render http://floodlist.com/news page it only shows a page with You are being redirected... title.

I found this issue so d-bus problem may be harmless.


Solution

  • These warnings/errors seems to be harmless (see #491 on Splash and #122 on scrapy-splash)

    And the problem with rendering was solved by increasing wait value up to 1 second. More info about rendering issues here.