Search code examples
selenium-webdriverselenium-chromedrivercloudflare

Selenium: Stuck in Cloudflares "Checking if the site connection is secure"

So since this week my selenium test fails because the cloudflare does not redirect me to the page i want to open.

Im seeing this: Cloudflare stucked

Im executing my selenium tests in incognito and these are the arguments im using:

ChromeOptions options = new();
options.AddArguments("--incognito");
options.AddArguments("--ignore-ssl-errors=yes");
options.AddArguments("--ignore-certificate-errors");
options.AddArguments("--disable-gpu");
options.AddArguments("--lang=en");
options.AddArguments("--disable-infobars");
options.AddArguments("--window-size=1920,1080");
options.AddArguments("--disable-blink-features=AutomationControlled");

When i open the website in normal mode or incognito mode, it works with no problems.

Only when it opens with selenium, i get stucked there.

I tried to use options.AddArguments("--disable-blink-features=AutomationControlled"); as mentioned in other answers, but this doesnt solve the problem.

Thanks!

Solution

  • As mentioned by @browsemator in the comment you just experienced Cloudflare's bot detection and I don't think that is is possible to easily avoid it. However there are at least two options you could try.

    Undetected Chrome Driver

    I assume you are using C# so you could use either this or this NuGet package to setup your driver. If you would use the first one, then the setup would look something like this:

    IWebDriver driver = UndetectedChromeDriver.Create(driverExecutablePath: "path\\to\\chromedriver.exe");

    There is an option (I don't remember with which package of these two) to install the chromedriver automatically without you needing to handle this yourself, but I could only make that work for latest versions of chromedriver and those do not include the actual latest ones needed for latest chrome.

    Chrome Profile

    You could use an existing Chrome profile. Your code would look something like this:

    ChromeOptions options = new ChromeOptions();
options.AddArgument($"--user-data-dir={userDataDir}");
options.AddArgument($"--profile-directory={profileDir}");
IWebDriver driver = new ChromeDriver(options);

    For userDataDir and profileDir you would need to specify the path for your specific Chrome profile which you can find by typing chrome://version in your browser. For an example we would see in the "Profile Path" section this path:

    C:\Users\YourName\AppData\Local\Google\Chrome\User Data\Some Profile

    With this example path the value for userDataDir would be C:\Users\YourName\AppData\Local\Google\Chrome\User Data and for profileDir the value would be Some Profile.

    This is how this approach would help:

    1. Open Chrome in the specific profile that will be used for automated test
    2. Navigate to the site manually
    3. Close the window of that specific profile (all the windows)
    4. Run the test

    When you open the website manually, a cookie is created which Cloudflare checks to verify that it is not a bot user. The cookie remains during the automated test as you are using that same specific Chrome profile. Keep in mind, that this might periodically fail as the cookie might expire or Cloudflare will do some additional checks or something and you will need to repeat the manual opening of the website from time to time.