Search code examples
javascriptselenium-webdriverxpathjs-scrollintoviewexecute-script

selenium.common.exceptions.InvalidSelectorException: Message: Given xpath expression is invalid using xpath within scrollIntoView through Selenium

I use Python to scrape a website with a filter pane that needs to be scrolled. I found a code that helps to scroll through the list of elements, that actually find a list and move through a loop.

recentList = driver.find_elements_by_xpath("/html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li")
for list in recentList:
    driver.execute_script('arguments[0].scrollIntoView({behavior: "smooth", block: "end", inline: "nearest"})', list)

I have code that already contains a for loop and I would like to add only an element that needs to be scrolled to. With the logic of the above I've written (a loop is simplified):

for p in range(1,15):
    list = driver.find_element_by_xpath(str('/html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li[[' + str(p) + ']'))
    driver.execute_script('arguments[0].scrollIntoView({behavior: "smooth", block: "end", inline: "nearest"})', list)

I can't figure out why it doesn't work. Here is an Error I got:

selenium.common.exceptions.InvalidSelectorException: Message: Given xpath expression "/html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li[[1]" is invalid: SyntaxError: The expression is not a legal expression.

Does anyone know what needs to be fixed? The XPath in the last code is correct and was already in use.

I tried to replace my current loop with the "list in recentList" but then code stops when it needs to scroll through pages on a filter.

Solution

  • This error message...

    selenium.common.exceptions.InvalidSelectorException: Message: Given xpath expression "/html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li[[1]" is invalid: SyntaxError: The expression is not a legal expression.

    ...implies that the XPath expression was not a valid/legal expression.

    If you observe the effective within the error, there is an extra third bracket opening within the expression:

    /html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li[[1]

    Solution

    To remove the extra third bracket opening, you need to adjust the xpath expression as:

    list = driver.find_element_by_xpath(str('/html/body/div[3]/main/div/div/div/div[1]/div/form/div[2]/fieldset[3]/div/ul/li[' + str(p) + ']'))

    References

    You can find a couple of relevant discussions in: