Search code examples
luascrapyscrapy-splash

Lua scripts select dropdown

I have a page I am practicing some scraping on and having trouble with the Lua script to select from a drop down.I have been trying for hours now to figure it out. I need to select 8.4 from the elementid right_eye_156 Here is what I have right now which doesnt do anything but select the title of the page.

function main(splash, args)
    assert(splash:go(splash.args.url))
    local element = splash:select('.element')
    return splash:evaljs("document.title")
end

here is the html for the dropdown from the url (https://eoptika.hu/termekek/kontaktlencse/acuvue-oasys-with-hydraclear-plus-6-db-1-2-heti-kontaktlencse.html)

<select class="form-control eye-option select01 mycol-xs-6 right-eye" id="right_eye_156">
    <option value="" selected="selected" class="choose">...</option>
    <option value="973" class="right">8.4</option>
    <option value="5526" class="right">8.8</option>              
</select>
Solution

  • Find xpath of the option that contains text '8.4' 

    local element_xpath = "//select[@id='right_eye_156']/option[contains(.,'8.4')]"

    Use splash:evaljs to find element using xpath 

    local element = splash:evaljs("document.evaluate(\"" .. element_xpath .. "\", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue")