Search code examples
spring-bootserializationinfinispan

Implementing Infinispan (Data Grid) onto existing Spring Boot application and face problems with encoding

I have a existing Spring Boot 3 application that runs single node with Caffeine cache and the @Cacheable annotations.

The app runs fine so far, as Caffeine doesn't care about the POJO structure or format, it will just store the keys and the values in memory. Remember that, in the Spring Cache world, keys are often objects, e.g. a cached Person getPerson(String firstName, String lastName) uses a compound key.

The application is business-critical with regards to performance. It operates in payment industry so I'll dedicate a few neurons to security considerations.

Okay, now it's time to implement Red Hat Data Grid on Openshift. So I fired up a local instance of Infinispan 14 and hoped that the auto-configuration did some magic.

Here is the infinispan.yaml

infinispan:
  local-cache-configuration:
    name: base-template
    expiration:
      lifespan: 1800000
    encoding:
      media-type: application/x-java-object #We'll talk about this
  cache-container:
    name: default
    caches:
      ipe.my-ipe-cache: # Consider it is referenced in @Cacheable(value = CACHE_KEY)
        local-cache:
          configuration: base-template
      # Consider 24 identical nodes
  server:
    interfaces:
      - name: public
        inet-address: 
         value: 127.0.0.1
    socket-bindings:
      default-interface: public
      port-offset: 0
      socket-binding:
        - name: default
          port: 11222
        - name: memcached
          port: 11221
    endpoints:
      endpoint:
        socket-binding: default
        security-realm: default
    security:
      security-realms:
        - name: default
          properties-realm:
            groups-attribute: Roles
            user-properties:
              path: users.properties
            group-properties:
              path: groups.properties

And application.yaml

infinispan:
  embedded:
    enabled: false
  remote:
    enabled: true
    server-list: "localhost:11222"
    auth-username: admin
    auth-password: admin
#    marshaller: org.infinispan.commons.marshall.UTF8StringMarshaller
#    marshaller: org.infinispan.commons.marshall.ProtoStreamMarshaller
    marshaller: org.infinispan.commons.marshall.JavaSerializationMarshaller
    java-serial-allow-list: "com\\.acme\\.ipe\\.(.*)\\.dto\\.\\w+"

Now the problem lies in configuring the correct marshalling, and I am stuck at this

  • Java serialization From what I understood, Java serialization is recommended only for embedded caches, and has all the drawbacks related to Java serialization, both for security and compatibility with other clients. For the security part, I acknowledge that I added (at least on Java client side) the allow list of the DTOs that will always contain "non-esoteric" attributes (e.g. all Strings, numbers, date-times and DTOs or list of DTOs with same features). As for the compatibility part, I acknowledge that a change in the code base may require a cache flush, which is not that derimental

Anyway it didn't work

org.infinispan.client.hotrod.exceptions.HotRodClientException: org.infinispan.commons.CacheException: ISPN000936: Class 'com.acme.ipe.core.data.dto.ChannelDto' blocked by deserialization allow list. Adjust the configuration serialization allow list regular expression to include this class.
    at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
    at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2096)
    at org.infinispan.client.hotrod.impl.Util.await(Util.java:52)
    at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:196)
    at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:186)
    at org.infinispan.spring.common.provider.SpringCache.put(SpringCache.java:145)
    at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:87)
  • Protocol buffers Super duper recommended by Infinispan. Problem is, as far as I could learn, every POJO has to be annotated with protocol buffer annotations. There is a large amount of work, in this case, as the POJOs have inheritance and dozens of attributes. In fact, if I try to run my application as it is, Infinispan complains it cannot find a Marshaller for the DTO
java.lang.IllegalArgumentException: No marshaller registered for object of Java type com.acme.ipe.core.data.dto.ChannelDto : ChannelDto(id=1, creationDate=2017-11-06T06:40:06, ...)
    at org.infinispan.protostream.impl.SerializationContextImpl.getMarshallerDelegate(SerializationContextImpl.java:503)
    at org.infinispan.protostream.WrappedMessage.writeMessage(WrappedMessage.java:281)
    at org.infinispan.protostream.WrappedMessage.write(WrappedMessage.java:242)
    at org.infinispan.protostream.ProtobufUtil.toWrappedByteBuffer(ProtobufUtil.java:153)
    at org.infinispan.commons.marshall.ImmutableProtoStreamMarshaller.objectToBuffer(ImmutableProtoStreamMarshaller.java:55)
    at org.infinispan.commons.marshall.AbstractMarshaller.objectToByteBuffer(AbstractMarshaller.java:70)
    at org.infinispan.client.hotrod.marshall.MarshallerUtil.obj2bytes(MarshallerUtil.java:117)
    at org.infinispan.client.hotrod.DataFormat$DataFormatImpl.valueToBytes(DataFormat.java:92)
    at org.infinispan.client.hotrod.DataFormat.valueToBytes(DataFormat.java:211)
    at org.infinispan.client.hotrod.impl.RemoteCacheImpl.valueToBytes(RemoteCacheImpl.java:610)
    at org.infinispan.client.hotrod.impl.RemoteCacheImpl.putAsync(RemoteCacheImpl.java:306)
    at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:196)
    at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:186)
    at org.infinispan.spring.common.provider.SpringCache.put(SpringCache.java:145)
    at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:87)
  • JSON? I couldn't sort a way to let Infinispan itself (perhaps the client side on Spring Boot) to marshal/unmarshal the DTO as JSON, which would instantly solve all issues.

I can set the media-type of the cache to application/json but I can't find any marshaller in my classpath that supports

Wrapping it up

Is it possible to configure Infinispan to marshal/unmarshal objects to JSON which would solve all possible security and compatibility issues?

Othwerise, is it possible not to manually annotate all existing DTOs with Protobuf annotations, e.g. by using some IDE or Maven plugin that will likely generate a mapping for you?

And as last resort for today, how can I overcome the error ISPN000936 shown a few lines ago? I did add the allow list to my Spring application!

Solution

  • The problem you are having with Java Serialization is that you are telling the server to use encoding: application/x-java-object. This means that the server will want to have the bytecode of your classes to be able to interpret the data and possibly transcode it. This is also the reason for the org.infinispan.commons.CacheException: ISPN000936: Class 'com.acme.ipe.core.data.dto.ChannelDto' blocked by deserialization allow list. error you are getting: it's coming from the server, not the client. Use application/octet-stream on the server to avoid this. Also, JBoss Marshalling will offer better performance than plain Java serialization.

    Protocol Buffers are obviously the ideal choice, but they do require a lot of legwork if adapting a lot of existing code.

    As for JSON, you could implement a client JSONMarshaller using Jackson that does the marshalling/unmarshalling. This is something that could also be contributed upstream.