Search code examples
luascrapylua-tablescrapy-splash

Lua: "error": "attempt to call method 'mouse_click' (a nil value)",


I am new to Lua and trying to iterate over all the links from "items" by using "for" loop and click all the links using the mouse_click() function. But it throws "error": "attempt to call method 'mouse_click' (a nil value)" How can I do that? I have tried the below-mentioned code by myself but there may be something missing. From the documentation, I found that Lua has two types of for loop. Numeric and Generic. But to iterate through the links and after that click on it what I have to use?

Once again, I am very new to Lua. Just trying to get familiar with it so I am seeking all the experts' guidance.

function main(splash, args)
  
  splash:set_user_agent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36")
  splash.private_mode_enabled = false
  assert(splash:go(args.url))
  assert(splash:wait(3))
  
  items = assert(splash:select("div.item-list-content a"))
  
  for item in pairs(items) do
    item:mouse_click()
    assert(splash:wait(5))
  end
  
  splash:set_viewport_full()
  return {
    html = splash:html(),
    png = splash:png(),
    har = splash:har(),
  }
end

Output

{
    "error": 400,
    "type": "ScriptError",
    "description": "Error happened while executing Lua script",
    "info": {
        "source": "[string \"function main(splash, args)\r...\"]",
        "line_number": 11,
        "error": "attempt to call method 'mouse_click' (a nil value)",
        "type": "LUA_ERROR",
        "message": "Lua error: [string \"function main(splash, args)\r...\"]:11: attempt to call method 'mouse_click' (a nil value)"
    }
}

Solution

  • Select returns only one element, so it doesn't make sense to iterate over it using pairs (as you would be iterating over that element's fields, which is probably not what you want). You probably want to use select_all and then use ipairs to iterate over the elements that match the selector. See the select documentation for details and examples.

    [Update to address your comment] If you look at the example I reference, you'll see that it looks like this:

        local imgs = splash:select_all('img')
        local srcs = {}
    
        for _, img in ipairs(imgs) do
          srcs[#srcs+1] = img.node.attributes.src
        end
    

    As ipairs returns two values (index and its value), you'll have to use for _, item in ipairs(items) instead of for item in ipairs(items) (as in the latter case item will get index value instead of the item).