Search code examples
javascriptpythonscreen-scrapingpyqtpyside

Outputting and Responding to Javascript with PyQt

I am trying to use PyQt to load the html of a web page which can then be manipulated and fed back to the page for web scraping. I am basically trying to log into a page with Javascript on it, search for documents to download (by selecting a check box next to the correct one's names), and then clicking a download button which pops out another page.

Does anyone know the functions I would use? Is there a way to discuss this without going into Classes? (My understanding of Classes is not as good as it could be, I am trying to learn, I'm still something of a beginner).

Sorry if I didn't explain this well. I'm trying to use either PyQt or PySide to do this.

Solution

  • I use pyqt/pyside to load a page, wait for the JavaScript to execute, then parse the resulting HTML for the content of interest.

    Here is an example script:
    http://webscraping.com/blog/Scraping-JavaScript-webpages-with-webkit/