I wrote a web scraping program on python (with selenium) which works very well. The last step was to add a headless functionality. The problem is that when the script is running headless, at a specific moment (when it should click over a link to refresh the page), terminal gives me this error:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <a href="javascript:void(0);" onclick="prtl.refresh(this);">...</a> is not clickable at point (217, 157). Other element would receive the click: <span ewnet:toggler="290" class="prtl">...</span>
The weird thing is that when the same script is not headless, selenium doesn't throw the error and all goes perfectly.
I have tried to reach the element using several methods but same error.
This is when the script decides if the browser is headless or not:
# create driver
if headless:
chromedriver = './chromedriver'
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(executable_path = chromedriver, chrome_options = options)
else:
driver = webdriver.Chrome('./chromedriver')
The element I'm trying to click looks something like this:
<a href='some_url' onclick='refresh(this)'>
<span class='name'>refresh page!</span>
</a>
and my script searches it by xpath on <a> tag.
The solution is to ensure that the window size is large enough for the objects to be clickable otherwise the window-size property is set to 0x0 and the objects will be covering each other:
options.add_argument('window-size=2000x1500')
So to code will look like this:
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('window-size=2000x1500')
This works for chromedriver. I have not checked other headless browsers, but expect the same or similar behavior.