How to select elements from an already selected element object with splash

Question

After selecting an element using splash:select how do you then select all the anchor elements found as children under it?

I have tried this lua script with scrapy/splash:

function main(splash)
    assert(splash:go(splash.args.url))
    assert(splash:wait(0.9))

    local classlist = splash:select('.class-list')        
    local alinks = classlist:select_all('a')

    return {alinks=alinks}

end

But I get the following bad request to Splash error:

{
    u'info':{
        u'line_number':12,
        u'message':        u'Lua error:[
            string "..."
        ]:12:attempt to call method \'select_all\' (a nil value)',
        u'type':u'LUA_ERROR',
        u'source':u'        [
            string "..."
        ]        ', u'        error':u"attempt to call method 'select_all' (a nil value)"
    },
    u'type':u'ScriptError',
    u'description':u'Error happened while executing Lua script',
    u'error':400
}

I have confirmed that the splash:select('.class-list') returns a valid element object.

Answer 1

As you say, splash:select('.class-list') returns a valid element object. Your problem is that element objects don't have a select_all method; only the splash object does. Instead you need to use splash:select_all with a different selector. Try the following (though beware, as it is untested).

function main(splash)
    assert(splash:go(splash.args.url))
    assert(splash:wait(0.9))
    local alinks = splash:select_all('.class-list a')        
    return {alinks=alinks}
end