Search code examples
spring-bootwebsphere-libertygemfirespring-data-gemfire

ClassLoader issues: GemFire JCA Resource Adapter in IBM WebSphere LibertyBase

I'm attempting to setup a JCA resource adapter for Gemfire 9.8 on IBM WebSphere Liberty base by following the link https://gemfire.docs.pivotal.io/98/geode/reference/archive_transactions/JTA_transactions.html#concept_cp1_zx1_wk

Here is my ra.xml:

<connector xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/connector_1_5.xsd"
version="1.5">

<display-name>GFE JCA Adaptor</display-name>
<vendor-name></vendor-name>
<spec-version>1.5</spec-version>
<eis-type>GFE JCA</eis-type>
<version>1.5</version>
<resourceadapter>
    <config-property>
        <config-property-name>ProductName</config-property-name>
        <config-property-type>java.lang.String</config-property-type>
        <config-property-value>GemFire</config-property-value>
    </config-property>
    <config-property>
        <config-property-name>UserName</config-property-name>
        <config-property-type>java.lang.String</config-property-type>
        <config-property-value/>
    </config-property>
    <config-property>
        <config-property-name>Version</config-property-name>
        <config-property-type>java.lang.String</config-property-type>
        <config-property-value>8.0</config-property-value>
    </config-property>

    <outbound-resourceadapter>
        <connection-definition>
            <managedconnectionfactory-class>org.apache.geode.internal.ra.spi.JCAManagedConnectionFactory</managedconnectionfactory-class>
            <connectionfactory-interface>org.apache.geode.ra.GFConnectionFactory</connectionfactory-interface>
            <connectionfactory-impl-class>org.apache.geode.internal.ra.GFConnectionFactoryImpl</connectionfactory-impl-class>
            <connection-interface>org.apache.geode.ra.GFConnection</connection-interface>
            <connection-impl-class>org.apache.geode.internal.ra.GFConnectionImpl</connection-impl-class>
            <transaction-support>LocalTransaction</transaction-support>
            <reauthentication-support>false</reauthentication-support>
        </connection-definition>
    </outbound-resourceadapter>
</resourceadapter>

And here is my resource adapter setting:

<library id="gemfireRaLib" apiTypeVisibility="spec, ibm-api, stable, third-party, api">
  <fileset dir="path/to/geode-lib" includes="geode-dependencies.jar"/>
</library> 

<resourceAdapter id="gemfireJCA" location="/path/to/geode-lib/geode-jca-9.8.3.rar">
    <classloader apiTypeVisibility="spec, ibm-api, stable, third-party, api" commonLibraryRef="gemfireRaLib" delegation="parentFirst"/>
</resourceAdapter>

When I start my liberty server, spring boot initialization fails saying ClassNotFoundException org.apache.geode.ra.GFConnectionFactory is not found.

Then I put all geode dependencies as a shared library:

<library id="gemfireRaLib" apiTypeVisibility="spec, ibm-api, stable, third-party, api">
  <fileset dir="/path/to/geode-lib" includes="geode-dependencies.jar"/>
  <fileset dir="/path/to/geode-lib" includes="*.jar"/>
</library> 

<resourceAdapter id="gemfireJCA" location="/path/to/geode-jca-9.8.3.rar">
    <classloader apiTypeVisibility="spec, ibm-api, stable, third-party, api" commonLibraryRef="gemfireRaLib" delegation="parentFirst"/>
</resourceAdapter>

and

<webApplication contextRoot="apprRoot13" location="/path/to/mylocation.war" name="App13" id="App13">
    <classloader apiTypeVisibility="spec, ibm-api, stable, third-party, api" commonProviderRef="gemfireRaLib" delegation="parentFirst" />
</webApplication>

Error:

spring-data-gemfire initialization fails:
AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from class path resource [applicationContext-gemfire.xml]; nested exception is java.lang.NoClassDefFoundError: org.apache.geode.cache.PartitionResolver

In all above scenario I get gemfireJCA adapter installed successfully.

Update 1

/path/to/shared/geode-lib/ contains the following jars:

geode-common-9.8.3.jar geode-cq-9.8.3.jar geode-jca-9.8.3.rar geode-management-9.8.3.jar geode-core-9.8.3.jar geode-dependencies.jar geode-lucene-9.8.3.jar geode-wan-9.8.3.jar

When I pack them inside WAR the application bootsup but fails at

lookup = (GFConnectionFactory) template.lookup("gfe/jca");

saying:

java.lang.ClassCastException: org.apache.geode.internal.ra.GFConnectionFactoryImpl incompatible with org.apache.geode.ra.GFConnectionFactory

I get java.lang.ClassCastException: org.apache.geode.internal.ra.GFConnectionFactoryImpl incompatible with org.apache.geode.ra.GFConnectionFactory because GFConnectionFactoryImpl and GFConnectionFactory are loaded by different classloaders. Thats why I created a geode-lib shared library. Sharing it between JCA resource adapter and web application

Solution

  • Thanks a lot for your response. Here is the root cause of the issue.

    • In Liberty there is no SingleClassloader per application policy. The classloader of a JCA resource adapter is different and WebApplication is different.
    • Hence geode-*.jar is loaded by 2 different classloaders.
    • GFConnectionFactory provided by gemfireJCA RA is different from application. Hence it is impossible to 'class cast' GFConnectionFactory loaded by JCA classloader into application classloader.

    To solve this I can follow any of the below solutions:

    Solution 1:

    Deploy GemfireJCA and WAR in a Single EAR. Pack geode-jca.rar along with geode-*.jar and geode-dependencies.jar as a common library and WAR.

    Please note: These geode-*.jar should not be in WEB-INF/lib of WAR. Here is the EAR tree: App13.ear: | |--gemfire-jca-9.8.3.rar |--WebApp13.war(Without geode-*.jar) |--lib | |--geode-common-9.8.3.jar | |--geode-cq-9.8.3.jar | |--geode-management-9.8.3.jar | |--geode-core-9.8.3.jar | |--geode-dependencies.jar | |--geode-lucene-9.8.3.jar | |--geode-wan-9.8.3.jar |--META-INF | |--application.xml (with module connector for gemfireJCA using gemfire-jca-9.8.3.rar)

    Solution 2:

    Handle connection via source code:

    @Bean
public JCAManagedConnectionFactory jcaManagedConnectionFactory() {

    JCAManagedConnectionFactory jcaManagedConnectionFactory =
        new JCAManagedConnectionFactory();
    jcaManagedConnectionFactory.setProductName("GemFire");
    jcaManagedConnectionFactory.setUserName(""); // DO NOT SET ANY USERNAME
    jcaManagedConnectionFactory.setVersion("8.0");
    return jcaManagedConnectionFactory;
}
@Bean
public JCAManagedConnection jcaManagedConnection(JCAManagedConnectionFactory jcaManagedConnectionFactory)
    throws ResourceException {

    return (JCAManagedConnection) jcaManagedConnectionFactory
        .createManagedConnection(null, null);
}
    @Bean
public GFConnectionFactory getGFConnectionFactory(JCAManagedConnectionFactory jcaManagedConnectionFactory,
                                                  JCAManagedConnection jcaManagedConnection)
    throws ResourceException {

    GFConnectionFactory lookup =
        (GFConnectionFactory) jcaManagedConnectionFactory
            .createConnectionFactory(new ConnectionManager() {

                private static final long serialVersionUID =
                    1L;

                @Override
                public Object allocateConnection(ManagedConnectionFactory mcf,
                                                 ConnectionRequestInfo cxRequestInfo)
                    throws ResourceException {

                    return jcaManagedConnection.getConnection(null,
                        cxRequestInfo);
                }
            });
    return lookup;
}

    Clean up connection during shutdown:

     @Override
public void onApplicationEvent(ContextClosedEvent event) {

    if (null != jcaManagedConnection) {
        try {
            jcaManagedConnection.cleanup();
        } catch (ResourceException e) {
            LOGGER.error("Error while shuttingdown the container {}",
                e.getMessage());
            LOGGER.error(e.getMessage(), e);
        }
    }
}