Search code examples
cluster-computingglassfish-3load-balancingajp

Glassfish instances unable to startup after enabling AJP

I'm trying to make a cluster with Glassfish 3.1.2.2, I created cluster, then instances and started them and every thing is ok, then I enabled AJP port to make loadbalancer from Apache with mod_jk with the following steps (from Glassfish side):

  • I got jar files (commons-logging-1.1.1.jar,commons-modeler-2.0.1.jar,tomcat-ajp.jar) from tomcat and put them inside lib folder of Glassfish (not domain).

  • Then I added system properties (with special values) for each instance :

    AJP_INSTANCE_NAME and AJP_PORT

  • finally to enable ports, in JVM Options for cluster config :

    -Dcom.sun.enterprise.web.connector.enableJK=${AJP_PORT} -DjvmRoute=${AJP_INSTANCE_NAME}

And tried to start instance using the following command :

./asadmin start-local-instance --sync=full --node=node1 ins1

And fail with the following logs : From console : 

Removing all cached state for instance ins1.
Waiting for ins1 to start ...................................................................................................Error starting instance ins1.
The server exited prematurely with exit code 0.
Before it died, it produced the following output:

Launching GlassFish on Felix platform
[#|2013-04-29T16:46:55.932+0300|INFO|oracle-glassfish3.1.2|com.sun.enterprise.server.logging.GFFileHandler|_ThreadID=1;_ThreadName=main;|Running GlassFish Version: Oracle GlassFish Server 3.1.2.2 (build 5)|#]

[#|2013-04-29T16:46:59.585+0300|INFO|oracle-glassfish3.1.2|org.glassfish.ha.store.spi.BackingStoreFactoryRegistry|_ThreadID=10;_ThreadName=main;|Registered org.glassfish.ha.store.adapter.cache.ShoalBackingStoreProxy for persistence-type = replicated in BackingStoreFactoryRegistry|#]

[#|2013-04-29T16:47:08.881+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.transaction.com.sun.jts.CosTransactions|_ThreadID=10;_ThreadName=main;|JTS5014: Recoverable JTS instance, serverId = [100]|#]

[#|2013-04-29T16:47:16.645+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=11;_ThreadName=Grizzly-kernel-thread(1);|Grizzly Framework 1.9.50 started in: 1259ms - bound to [0.0.0.0:28181]|#]

[#|2013-04-29T16:47:16.646+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=12;_ThreadName=Grizzly-kernel-thread(1);|Grizzly Framework 1.9.50 started in: 1872ms - bound to [0.0.0.0:28080]|#]

[#|2013-04-29T16:47:16.833+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=13;_ThreadName=Grizzly-kernel-thread(1);|Grizzly Framework 1.9.50 started in: 219ms - bound to [0.0.0.0:24848]|#]

[#|2013-04-29T16:47:17.308+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=14;_ThreadName=Grizzly-kernel-thread(1);|Grizzly Framework 1.9.50 started in: 175ms - bound to [0.0.0.0:27676]|#]

[#|2013-04-29T16:47:24.329+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=main;|WEB0169: Created HTTP listener [http-listener-1] on host/port [0.0.0.0:28080]|#]

[#|2013-04-29T16:47:24.587+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=main;|WEB0169: Created HTTP listener [http-listener-2] on host/port [0.0.0.0:28181]|#]

[#|2013-04-29T16:47:24.812+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=main;|WEB0169: Created HTTP listener [admin-listener] on host/port [0.0.0.0:24848]|#]

[#|2013-04-29T16:47:24.998+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=main;|WEB0170: Apache mod_jk/jk2 attached to virtual-server [server]  listening on port [9,090]|#]

[#|2013-04-29T16:47:25.058+0300|SEVERE|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.server|_ThreadID=10;_ThreadName=main;|Startup service failed to start : com.sun.enterprise.v3.services.impl.WebContainerStarter|#]

[#|2013-04-29T16:47:26.791+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.tools.admin.com.sun.enterprise.v3.admin.cluster|_ThreadID=15;_ThreadName=Thread-19;|Server shutdown initiated|#]

[#|2013-04-29T16:47:26.879+0300|INFO|oracle-glassfish3.1.2|javax.enterprise.system.core.com.sun.enterprise.v3.server|_ThreadID=15;_ThreadName=Thread-19;|Already stopped, so just returning|#]



Command start-local-instance failed.

Edit: I put first logs before I disable GMS for test, I disabled it and still got log in console with failed message.

I have Solaris x86 OS, JDK is 1.6u37 What I did here I did it before in 2.1.1 and was working well with no issues, but with GF 3.1.2.2 I have no idea whats going on now.

Solution

  • I just wonder why I work days in issue, then when I write a question I find a solution, funny. Anyway I found my solution but not tested yet, but at least I saw ports in "LISTEN" status:

    The problem is from "-Dcom.sun.enterprise.web.connector.enableJK=${AJP_PORT}" seems not the right way anymore, its in Glassfish 2.1 but in Glassfish 3.1.2 you need to :

    1. From your cluster config --> Network Config --> Network Listeners (click on this)
    2. Create "New..." listener.
    3. Fill new listener info :
      • Name it like : jk-connector , port with: ${AJP_PORT} , check on "JK Listener" to enable it , for protocol select "Use an existing protocol" then select "http-listener-1" protocol , for Thread Pool select "http-thread-pool" (not sure about this till now, but used same pool of another listeners).

    Then start instances, if started successfully try check their ports is listening using "netstat -an | grep " eg: "netstat -an | grep 9090" and so.