Search code examples
pythonluasyntax-errorscrapy-splashsplash-js-render

Error expected near on Splash Lua Script


I need to run this simple Lua script on Splash:

    script = """
    function main(splash)
        local url = splash.args.url
        assert(splash:go(url))
        assert(splash:wait(1))

        -- go back 1 month in time and wait a little (1 second)
        assert(splash:runjs("$('a[data-ci-test-id=\"checkOutButton\"]').click()"))
        assert(splash:wait(1))

        -- return result as a JSON object
        return {
            html = splash:html(),
            -- we don't need screenshot or network activity
            --png = splash:png(),
            --har = splash:har(),
        }
    end
    """

But I got this error:

{u'info': {u'source': u'[string "..."]', u'message': u'[string "..."]:8: \')\' expected near \'checkOutButton\'', u'line_number': 8, u'type': u'LUA_INIT_ERROR', u'error': u"')' expected near 'checkOutButton'"}, u'type': u'ScriptError', u'description': u'Error happened while executing Lua script', u'error': 400}

How should I write the CSS selector string for this script to work?


Solution

  • I found the answer based on another answer even though the error string is not the same. Use scrapy + splash return html

    It turns out, the page I scrape had no JQuery, so I have to load the JQuery package first.

        script = """
        function main(splash)
            # add this vvv
            assert(splash:autoload("https://code.jquery.com/jquery-3.1.1.min.js"))
            local url = splash.args.url
            assert(splash:go(url))
            assert(splash:wait(1))
    
            -- go back 1 month in time and wait a little (1 second)
            assert(splash:runjs("$('div.minicart_summery > a.btn_checkout').click()"))
            assert(splash:wait(1))
    
            -- return result as a JSON object
            return {
                html = splash:html(),
                -- we don't need screenshot or network activity
                --png = splash:png(),
                --har = splash:har(),
            }
        end
        """