For a few years, we used our Ruby code running on some Heroku nodes utilizing Chrome browser driven by Selenium & its Chromedriver. Everything worked fine, until a week ago, at which point the code started failing to create a new Chrome instance. Read the long and convoluted detective story on...
(...a few hours of investigating later...) It turns out that when Heroku compiled the slug upon code release, heroku-buildpack-chromedriver pulled the most recent Chrome version, which was 117.0.5938.88, and when attempting to run, that binary failed by not finding a required library:
/app/.cache/selenium/chrome/linux64/117.0.5938.92/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory
(The Heroku stack we are are using is heroku-22 which is the most recent. There's no newer version... so "upgrade your stack!" is not a correct answer.)
Okay, so "you said that it worked before! well, then roll back to the previous version!". Indeed, tried that. I wouldn't be here asking this question otherwise. I do not tend to ask easy questions!
So, fine, on the previous version of the slug that worked (and still works, we had to roll back to that slug) fine, the versions are as follows:
> puts `which google-chrome`
/app/.apt/usr/bin/google-chrome
> puts `/app/.apt/usr/bin/google-chrome --version`
Google Chrome 115.0.5790.170 unknown
> Selenium::WebDriver::VERSION
=> "4.10.0"
> puts `which chromedriver`
/app/.chromedriver/bin/chromedriver
> puts `/app/.chromedriver/bin/chromedriver --version`
ChromeDriver 114.0.5735.90 (386bc09e8f4f2e025eddae123f36f6263096ae49-refs/branch-heads/5735@{#1052})
Fine, so I need to lock all of these to the versions listed above. Well, problem #1: https://github.com/heroku/heroku-buildpack-google-chrome doesn't allow you to pick a specific version, it only lets you to pick the "channel" and then will always give you the most recent version pulled from that channel. That's fine... I can't be the only one with this need, right? There must be some poor chap who ran into the same issue before, right? Of course there was one. Let's grab his buildpack, set the env var (CHROME_VERSION) with desired verison number... yay, we are rolling! ...or are we? Somehow, despite the fact that the buildpack works and the reqested Chrome version appears on the list, the old (or, rather, the newer Chrome version that we are trying to get rid of) still appears on the list of packages being installed, and as it gets its turn to install, it overwrites the older version that we want:
remote: -----> Installing google-chrome-stable_115.0.5790.170-1_amd64.deb
remote: -----> Installing google-chrome-stable_116.0.5845.140-1_amd64.deb
(...a few hours of investigating later...) It turns out that Heroku has a package cache! That doesn't get properly updated when an older package is installed - the "more modern" one isn't cleared from it, even though it is no longer needed. Fine, we'll modify the buildpack to forcefully purge all versions of google-chrome from the cache. Take that!
It should work now. Right? Riiiiight? Well, no wai! When we try to instantiate Selenium:
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless=new')
options.add_argument('disable-dev-shm-usage')
options.add_preference(:plugins, { always_open_pdf_externally: true })
options.add_preference(:download, { prompt_for_download: false })
http_client = Selenium::WebDriver::Remote::Http::Default.new
http_client.read_timeout = 300
@driver = Selenium::WebDriver.for(:chrome, options: options, http_client: http_client)
.... what happens? SERIOUSLY? Instead of picking up the browser that we spent so much time getting to download, freaking Selenium stubbornly downloads its own copy instead of using the existing one!!!
Well, okay, you don't know there already is a browser right here! Right? After all, which google-chrome gives you the right location! What do you mean that's not what you are looking for?! Well, okay, stop fumbling around and use EXACTLY this one!
Selenium::WebDriver::Chrome.path = "/app/.apt/opt/google/chrome/google-chrome"
...nope, not so easy. Still downloads the "newer" version. Which does not work (missing library, remember?)
At which point I am at loss. How exactly do I tell the damn thing to use exactly the version(s) I want? Or, alternatively, how do I get to install that damn library that somehow went AWOL (see the very beginning of this long, long page).
So, the problem was that Selenium refused to start. After fiddling for over a week trying to lock the Chrome version to an older one that worked (and still works) fine, I unlocked the version in hopes that this breakage was caused by some changes in Chrome. The problem with missing library indeed disappeared but another one appeared, with Chrome complaining about some issues with sandboxing. Well, fortunately, THAT problem disappeared with:
myopts = Selenium::WebDriver::Chrome::Options.new
myopts.add_argument('no-sandbox')
driver = Selenium::WebDriver.for(:chrome, options: myopts)
Everything works now so this question as it was written is now moot.