Search code examples
javac++jenkinsdeadlockeclipse-cdt

Java deadlock with Eclipse CDT headless build

My case is similar to this one: Eclipse CDT Headless build hangs after build is finished, but I digged a little bit into it and found other problems.

Scenario: I have a C++ application and a Jenkins build server on Linux (Debian Jessie 8.2). I have Eclipse CDT in my dev environment and it perfectly builds my C++ App (local build). I installed Eclipse CDT on the build server and my Jenkins script runs a headlessbuild to build my App.

Problem: Jenkins hangs even after the build finishes. But that's not Jenkins fault. I removed Jenkins from the equation and did an Eclipse headlessbuild in the build server and the result was the same, it hangs after printing out "Build Finished". It's worth noting that I can successfully build my App if I open Eclipse CDT in the build server and trigger a build using the UI.

Versions: In the build server, I tried Java 1.7 and 1.8. As for Eclipse, I tried Kepler, Luna, Mars and Neon (this last one required Java 1.8). For this post, I'm using this:

java -version
java version "1.7.0_111"
OpenJDK Runtime Environment (IcedTea 2.6.7) (7u111-2.6.7-1~deb8u1)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)

Commands: Here is the Eclipse CDT command for the headless build that I used:

~/eclipse/eclipse -nosplash -application org.eclipse.cdt.managedbuilder.core.headlessbuild -import ~/myAppProj/ -build myAppProj/Release -data ~/workspace/

Observations:

  1. While Java is hanging, I can kill its process and get the executable ready to use. When I trigger the build from Jenkins and kill that deadlocked Java process, Jenkins just continues running normally further scripts.

  2. In the build server, while Java is janging, if I run:

jstack [java pid]

Then I get the following:

Attaching to process ID 25262, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.111-b01
Deadlock Detection:

java.lang.RuntimeException: Unable to deduce type of thread from address 0x00007fc5c8032800 (expected type JavaThread, CompilerThread, ServiceThread, JvmtiAgentThread, or SurrogateLockerThread)

Conclusion: it does seem that Java is deadlocking when Eclipse is run in headless mode. It doesn't happen all the time, but 99% I guess. Also, it doesn't stay hanging forever, it comes back after an hour or two, which is bad for any build server.

So, did anybody have problems like that?

Solution

  • (As confirmed in the comments)

    The problem you are facing is likely a lock-up in the CDT indexer. Sometimes when the life cycle is short in Eclipse the CDT indexer unfortunately locks up. At the command line you can disable it by using the -no-indexer command line option (added in Eclipse Mars).

    The lockup is itself filed as a bug, See Bug 327126 - Deadlock attempting to cancel PDOMIndexerJob during a resource delta notification

    Your jstack problem is most likely unrelated and is likely due to mismatch in permissions, users, or JVMs between the java process and the jstack process.