Search code examples
javadebuggingselenium-chromedriver

How to unfreeze debugger while using Selenium ChromeDriver?


Tested on versions as:

  1. Java 11.0.9 (Zulu)

  2. Selenium-chrome-driver: 3.0.0, 3.0.1, 3.1.0, 3.2.0, 3.3.0, 3.3.1, 3.4.0, 3.5.1, 3.5.2, 3.5.3, 3.6.0, 3.7.0, 3.7.1 ... up to 3.141.59, ... 4.9.0.

  3. WebDriverManager 3.8.1 & 3.6.2.

I'm working with the following snippet of the code as provided below:

Some part of code I skipped for representing better view.

import com.mongodb.client.model.IndexOptions;
import com.mongodb.client.model.Indexes;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.bson.Document;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import project.DatabaseConnections;
import java.util.Date;
import java.util.List;
import java.util.concurrent.TimeUnit;
import static com.mongodb.client.model.Filters.*;

public class GBFeedReader implements Runnable {

    public void run() {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--headless");
        options.addArguments("--disable-gpu");
        options.addArguments("--disable-extensions");
        WebDriverManager.chromedriver().setup();

        ChromeDriver driver = new ChromeDriver(options); // the debugger is freezing after this specific line 
        driver.get("https://gearbest.net");
        WebElement ulElement = driver
                .findElement(By
                        .xpath("/html/body/div[12]/div/div/div/div[2]/div/ul/li[2]"));
        List<WebElement> liElements = ulElement.findElements(By.tagName("div"));

            // class of the whole element
            liElements.forEach(liElement -> {
                //String str = "";
               /* Element aElement = liElement.select("a").first(); // header of element
                String url = aElement.attr("href");
                String title = aElement.attr("title"); // title of element
                Element spanElement = liElement.select("span").first();
                String price = spanElement.attr("data-mall-price"); // price of element*/
                ...
             });
         }
    }

My POM file is as below:

POM file

What I have already tried:

  1. I have tried to comment fully block of options:

     options.addArguments("--headless");
     options.addArguments("--disable-gpu");
     options.addArguments("--disable-extensions");
    

and re-debugg this code.

  1. I have tried to use versions in POM for Maven dependency as:

     <dependency>
         <groupId>org.seleniumhq.selenium</groupId>
         <artifactId>selenium-java</artifactId>
         <version>3.141.59</version>
     </dependency>
    

and:

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>4.9.0</version>
    </dependency>

and still the debugger is freezing.

  1. I have tried to go deeply inside business logic of the class ChromeDriver and see how it goes under the hood, so what I see is:

in DriverService.class the debugger shows as:

    public DS build() {
        if (this.port == 0) {
            this.port = PortProber.findFreePort();
        }

        if (this.exe == null) {
            this.exe = this.findDefaultExecutable();
        }

        ImmutableList<String> args = this.createArgs(); // args: size = 1
        return this.createDriverService(this.exe, this.port, args, this.environment); // port: 48393 exe: "C:\Users\invzbl3\.m2\repository\webdriver\chromedriver\win32\114.0.5735.90\chromedriver.exe"
    }

enter image description here

  1. I have checked similar questions as, for instance:

Chrome 'paused in debugger' issue
Paused in debugger in chrome?

but it doesn't look as my use case or I don't fully understand how it helps me here.

At the same time, my output in the console is as below:

"C:\Program Files\Java\zulu11.43.55-ca-jdk11.0.9.1-win_x64\bin\java.exe" -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:62518,suspend=y,server=n -javaagent:C:\Users\invzbl3\AppData\Local\JetBrains\IntelliJIdea2022.3\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath "E:\IdeaProject\WGIbot\target\classes;C:\Users\invzbl3\.m2\repository\org\telegram\telegrambots\4.2\telegrambots-4.2.jar;C:\Users\invzbl3\.m2\repository\org\telegram\telegrambots-meta\4.2\telegrambots-meta-4.2.jar;C:\Users\invzbl3\.m2\repository\com\google\inject\guice\4.2.2\guice-4.2.2.jar;C:\Users\invzbl3\.m2\repository\javax\inject\javax.inject\1\javax.inject-1.jar;C:\Users\invzbl3\.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.9.0\jackson-annotations-2.9.0.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\jaxrs\jackson-jaxrs-json-provider\2.9.8\jackson-jaxrs-json-provider-2.9.8.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\jaxrs\jackson-jaxrs-base\2.9.8\jackson-jaxrs-base-2.9.8.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\module\jackson-module-jaxb-annotations\2.9.8\jackson-module-jaxb-annotations-2.9.8.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.9.8\jackson-databind-2.9.8.jar;C:\Users\invzbl3\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.9.8\jackson-core-2.9.8.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\media\jersey-media-json-jackson\2.25.1\jersey-media-json-jackson-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\core\jersey-common\2.25.1\jersey-common-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\bundles\repackaged\jersey-guava\2.25.1\jersey-guava-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\osgi-resource-locator\1.0.1\osgi-resource-locator-1.0.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\ext\jersey-entity-filtering\2.25.1\jersey-entity-filtering-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\containers\jersey-container-grizzly2-http\2.25.1\jersey-container-grizzly2-http-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\external\javax.inject\2.5.0-b32\javax.inject-2.5.0-b32.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\grizzly\grizzly-http-server\2.3.28\grizzly-http-server-2.3.28.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\grizzly\grizzly-http\2.3.28\grizzly-http-2.3.28.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\grizzly\grizzly-framework\2.3.28\grizzly-framework-2.3.28.jar;C:\Users\invzbl3\.m2\repository\javax\ws\rs\javax.ws.rs-api\2.0.1\javax.ws.rs-api-2.0.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\core\jersey-server\2.25.1\jersey-server-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\core\jersey-client\2.25.1\jersey-client-2.25.1.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\jersey\media\jersey-media-jaxb\2.25.1\jersey-media-jaxb-2.25.1.jar;C:\Users\invzbl3\.m2\repository\javax\annotation\javax.annotation-api\1.2\javax.annotation-api-1.2.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\hk2-api\2.5.0-b32\hk2-api-2.5.0-b32.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\hk2-utils\2.5.0-b32\hk2-utils-2.5.0-b32.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\external\aopalliance-repackaged\2.5.0-b32\aopalliance-repackaged-2.5.0-b32.jar;C:\Users\invzbl3\.m2\repository\org\glassfish\hk2\hk2-locator\2.5.0-b32\hk2-locator-2.5.0-b32.jar;C:\Users\invzbl3\.m2\repository\org\javassist\javassist\3.20.0-GA\javassist-3.20.0-GA.jar;C:\Users\invzbl3\.m2\repository\javax\validation\validation-api\1.1.0.Final\validation-api-1.1.0.Final.jar;C:\Users\invzbl3\.m2\repository\org\json\json\20180813\json-20180813.jar;C:\Users\invzbl3\.m2\repository\org\apache\httpcomponents\httpclient\4.5.3\httpclient-4.5.3.jar;C:\Users\invzbl3\.m2\repository\org\apache\httpcomponents\httpcore\4.4.6\httpcore-4.4.6.jar;C:\Users\invzbl3\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\invzbl3\.m2\repository\commons-codec\commons-codec\1.9\commons-codec-1.9.jar;C:\Users\invzbl3\.m2\repository\org\apache\httpcomponents\httpmime\4.5.3\httpmime-4.5.3.jar;C:\Users\invzbl3\.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-java\3.141.59\selenium-java-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-api\3.141.59\selenium-api-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-chrome-driver\3.141.59\selenium-chrome-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-edge-driver\3.141.59\selenium-edge-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-firefox-driver\3.141.59\selenium-firefox-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-ie-driver\3.141.59\selenium-ie-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-opera-driver\3.141.59\selenium-opera-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-remote-driver\3.141.59\selenium-remote-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-safari-driver\3.141.59\selenium-safari-driver-3.141.59.jar;C:\Users\invzbl3\.m2\repository\org\seleniumhq\selenium\selenium-support\3.141.59\selenium-support-3.141.59.jar;C:\Users\invzbl3\.m2\repository\net\bytebuddy\byte-buddy\1.8.15\byte-buddy-1.8.15.jar;C:\Users\invzbl3\.m2\repository\org\apache\commons\commons-exec\1.3\commons-exec-1.3.jar;C:\Users\invzbl3\.m2\repository\com\google\guava\guava\25.0-jre\guava-25.0-jre.jar;C:\Users\invzbl3\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\invzbl3\.m2\repository\org\checkerframework\checker-compat-qual\2.0.0\checker-compat-qual-2.0.0.jar;C:\Users\invzbl3\.m2\repository\com\google\errorprone\error_prone_annotations\2.1.3\error_prone_annotations-2.1.3.jar;C:\Users\invzbl3\.m2\repository\com\google\j2objc\j2objc-annotations\1.1\j2objc-annotations-1.1.jar;C:\Users\invzbl3\.m2\repository\org\codehaus\mojo\animal-sniffer-annotations\1.14\animal-sniffer-annotations-1.14.jar;C:\Users\invzbl3\.m2\repository\com\squareup\okio\okio\1.14.0\okio-1.14.0.jar;C:\Users\invzbl3\.m2\repository\io\github\bonigarcia\webdrivermanager\3.8.1\webdrivermanager-3.8.1.jar;C:\Users\invzbl3\.m2\repository\org\slf4j\slf4j-api\1.7.25\slf4j-api-1.7.25.jar;C:\Users\invzbl3\.m2\repository\com\google\code\gson\gson\2.8.5\gson-2.8.5.jar;C:\Users\invzbl3\.m2\repository\org\rauschig\jarchivelib\1.0.0\jarchivelib-1.0.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\commons\commons-compress\1.18\commons-compress-1.18.jar;C:\Users\invzbl3\.m2\repository\org\slf4j\slf4j-log4j12\1.7.5\slf4j-log4j12-1.7.5.jar;C:\Users\invzbl3\.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;C:\Users\invzbl3\.m2\repository\org\jsoup\jsoup\1.8.3\jsoup-1.8.3.jar;C:\Users\invzbl3\.m2\repository\com\vdurmont\emoji-java\4.0.0\emoji-java-4.0.0.jar;C:\Users\invzbl3\.m2\repository\joda-time\joda-time\2.10.1\joda-time-2.10.1.jar;C:\Users\invzbl3\.m2\repository\org\apache\commons\commons-lang3\3.5\commons-lang3-3.5.jar;C:\Users\invzbl3\.m2\repository\org\mongodb\mongodb-driver-sync\3.11.0\mongodb-driver-sync-3.11.0.jar;C:\Users\invzbl3\.m2\repository\org\mongodb\bson\3.11.0\bson-3.11.0.jar;C:\Users\invzbl3\.m2\repository\org\mongodb\mongodb-driver-core\3.11.0\mongodb-driver-core-3.11.0.jar;C:\Users\invzbl3\.m2\repository\com\squareup\okhttp3\okhttp\3.14.0\okhttp-3.14.0.jar;C:\Users\invzbl3\.m2\repository\org\telegram\telegrambots-chat-session-bot\4.3.1\telegrambots-chat-session-bot-4.3.1.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-core\1.4.0\shiro-core-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-lang\1.4.0\shiro-lang-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-cache\1.4.0\shiro-cache-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-crypto-hash\1.4.0\shiro-crypto-hash-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-crypto-core\1.4.0\shiro-crypto-core-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-crypto-cipher\1.4.0\shiro-crypto-cipher-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-config-core\1.4.0\shiro-config-core-1.4.0.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-config-ogdl\1.4.0\shiro-config-ogdl-1.4.0.jar;C:\Users\invzbl3\.m2\repository\commons-beanutils\commons-beanutils\1.9.3\commons-beanutils-1.9.3.jar;C:\Users\invzbl3\.m2\repository\commons-collections\commons-collections\3.2.2\commons-collections-3.2.2.jar;C:\Users\invzbl3\.m2\repository\org\apache\shiro\shiro-event\1.4.0\shiro-event-1.4.0.jar;C:\Program Files\JetBrains\IntelliJ IDEA 2020.2.3\lib\idea_rt.jar" project.Main
Connected to the target VM, address: '127.0.0.1:62518', transport: 'socket'
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/C:/Users/invzbl3/.m2/repository/com/google/inject/guice/4.2.2/guice-4.2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Starting ChromeDriver 114.0.5735.90 (386bc09e8f4f2e025eddae123f36f6263096ae49-refs/branch-heads/5735@{#1052}) on port 33800
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
Starting ChromeDriver 114.0.5735.90 (386bc09e8f4f2e025eddae123f36f6263096ae49-refs/branch-heads/5735@{#1052}) on port 12359
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
ChromeDriver was started successfully.

Personally, I'm still looking for any ideas how to solve it, if I find something as a solution, I'll update this question with own results & attempts, but, as for now, the problem still exists.

Thank you in advance for any helpful recommendations or thoughts.

I'll apreciate it, colleagues :)


Solution

  • Option #1:

    The question was resolved thanks to the colleague's recommendation in the following manner:

    public void run() {
        ChromeOptions options = new ChromeOptions();
    
        WebDriverManager.chromedriver().browserVersion("118.0.5993.118").setup(); 
    

    // after changing version from 114.0.5735.90 (in the C:\Users\invzbl3\.m2\repository\webdriver\chromedriver\win32 folder)

    to 118.0.5993.118 manually.

        options.addArguments("--headless");
        options.addArguments("--disable-gpu");
        options.addArguments("--disable-extensions");
        WebDriverManager.chromedriver().setup();
    
        ChromeDriver driver = new ChromeDriver(options);
        driver.get("https://gearbest.net");
        WebElement ulElement = driver
                .findElement(By
                        .xpath("/html/body/div[12]/div/div/div/div[2]/div/ul/li[2]"));
        List<WebElement> liElements = ulElement.findElements(By.tagName("div"));
    
        // class of the whole element
        liElements.forEach(liElement -> {
            //String str = "";
               /* Element aElement = liElement.select("a").first(); // header of element
                String url = aElement.attr("href");
                String title = aElement.attr("title"); // title of element
                Element spanElement = liElement.select("span").first();
                String price = spanElement.attr("data-mall-price"); // price of element*/
                ...            
            });
        }
    }
    

    with dependency accordingly as:

    <!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
    <dependency>
        <groupId>io.github.bonigarcia</groupId>
        <artifactId>webdrivermanager</artifactId>
        <version>5.6.0</version>
    </dependency>
    

    versions as: 3.8.1 & 5.3.2 are also suitable per provided requirements.

    To be noticed:

    It works without any problem, but I don't have this specific version in the folder while setupping it manually, for some reason:

    enter image description here

    So, if someone finds a way how to setup it manually to get updated version of the Chromedriver in the folder directly, as well, it would be great.


    Option #2:

    Check the version of the dependency & downgrade it to lower one, as for instance: from 4.x.x to 3.x.x.

    Instead of:

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.x.x</version>
        </dependency>
    

    to:

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.x.x</version>
        </dependency>
    

    UPD:

    To clarify:

    I suppose it happens due to differences in internal instructions of different dependency versions, but what exactly it hards to say, at the current moment.