Search code examples
javascriptpythonscrapyscrapy-splash

scrapy javascript with spalsh wont render page

I want to crawl this page, I follow this post to crawl it but it didn't render webpage.
How can I fix it?
I use this:

scrapy shell 'http://localhost:8050/render.html?url=https://shenoto.com/album
/46321?lang=en&timeout=10&wait=0.5'

and simply

response.body

But it didn't render the page. and show only javascripts in page.
Actually I want to crawl "application/ld+json" part of that website as show in picture below.

image

Best regards

Solution

  • I found this post and fix the render issue.

    and this post to load ld-json.