Short version
I am running Selenium 3.11.0 + Geckodriver 0.18.0 + Firefox 59.0.2 on Ubuntu 14.04.5. When trying to open an URL, Selenium sometimes throws the following error
[Errno 111] Connection refused https://www.example.com
Geckodriver.log shows
1523165125053 geckodriver INFO geckodriver 0.18.0
1523165125060 geckodriver INFO Listening on 127.0.0.1:37893
1523165126313 geckodriver::marionette INFO Starting browser /usr/bin/firefox with args ["-marionette"]
1523165127142 Marionette INFO Enabled via --marionette
(firefox:28158): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(firefox:28158): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
(firefox:28158): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(firefox:28158): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
(firefox:28158): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(firefox:28158): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(/usr/lib/firefox/firefox:28235): GLib-GObject-CRITICAL **: g_object_unref: assertion 'object->ref_count > 0' failed
1523165129816 Marionette INFO Listening on port 41316
1523165129876 Marionette WARN TLS certificate errors will be ignored for this session
ATTENTION: default value of option force_s3tc_enable overridden by environment.
JavaScript error: jar:file:///usr/lib/firefox/omni.ja!/components/PeerConnection.js, line 1816: TypeError: this._dompc._pc is undefined
[Child 28235, Chrome_ChildThread] WARNING: pipe error (16): Connection reset by peer: file /build/firefox-4kFFck/firefox-59.0.2+build1/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
[Child 28235, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /build/firefox-4kFFck/firefox-59.0.2+build1/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
What's going on?
Long version:
I have a set of nightly jobs scheduled in cron. They are started with individual shell scripts that look like this
if pgrep "Xvfb" > /dev/null
then
echo "Xvfb already running"
else
Xvfb :99 -screen 0 1024x768x24 2>&1 >/dev/null &
fi
export DISPLAY=:99
python dostuff.py
There are several shell resembling the one above. Each of them starts a Python2.7 script (e.g. dostuff.py in the example above) that looks like this (the only difference being different set of urls)
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
for j in list_of_urls:
driver_get(j)
# do stuff to j
driver.quit()
This works ok most of the time, but occasionally the script becomes nonresponsive and I get a connection refused error. This seems to always happen in the middle of the loop. I.e. urls[0] is opened ok, but urls[1] might fail.
When googling for the error message found from geckodriver.log, I found this https://github.com/mozilla/geckodriver/issues/828. Since I am not maximising any windows, I do not think my problem is the same as described in the ticket (despite the error msg's showing similarity).
I am running several firefox instances (and all of them are using the same DISPLAY=:99). Could it be that this leads to the different Firefox instances interfering with one another?
I should also mention that this setup worked perfectly when I was using Firefox 44 (which did not require Geckodriver). Another thing that might be important is that as far as I can tell, this only happens when I am running the scripts with headless browsers using Xvfb.
As you can see from my questions, I am slightly out of my depth here with Xvfb and Geckodriver and Firefox. If someone has any idea what's going on, I would appreciate it.
Update: updating to Geckodriver 0.20.1 solved the problem - sort of.
The problem still appears (around once for every 100 pages opened). driver.get(j) occasionally fails, but it raises a timeout error which can be caught and handled.