How to save cached Keycloak data to a persistent data store?

Question

We are running Keycloak (v4.4, standalone mode) inside of 2 Docker containers. We wish these containers to be stateless, so we must persist all cached data to a backing store (either database or other caching solution such as Redis). We can not allow cached data to only exist in-memory, because either of our containers may be destroyed at any time.

Ideally, we would like to persist cached data to our own Redis instance. Since Keycloak uses Infinispan, it seems like this is the way to configure Infinispan to use Redis: http://infinispan.org/docs/cachestores/redis/.

Naively, I tried to have Keycloak store session information in Redis by updating my standalone-4.4.0.xml file to look like this (notice the redis-store element on line 5):

<subsystem xmlns="urn:jboss:domain:infinispan:6.0">
        <cache-container name="keycloak">
            <local-cache name="sessions">
                <persistence passivation="false">
                    <redis-store xmlns="urn:infinispan:config:store:redis:8.0"
                        topology="server" socket-timeout="10000" connection-timeout="10000">
                        <redis-server host="server1" />
                        <connection-pool min-idle="6" max-idle="10" max-total="20" min-evictable-idle-time="30000" time-between-eviction-runs="30000" />
                    </redis-store>
                </persistence>
            </local-cache>
            <local-cache name="realms">
                <object-memory size="10000"/>
            </local-cache>
            <local-cache name="users">
                <object-memory size="10000"/>
            </local-cache>
            <local-cache name="authenticationSessions"/>
            <local-cache name="offlineSessions"/>
            <local-cache name="clientSessions"/>
            <local-cache name="offlineClientSessions"/>
            <local-cache name="loginFailures"/>
            <local-cache name="work"/>
            <local-cache name="authorization">
                <object-memory size="10000"/>
            </local-cache>
            <local-cache name="keys">
                <object-memory size="1000"/>
                <expiration max-idle="3600000"/>
            </local-cache>
            <local-cache name="actionTokens">
                <object-memory size="-1"/>
                <expiration max-idle="-1" interval="300000"/>
            </local-cache>
        </cache-container>
        <cache-container name="server" default-cache="default" module="org.wildfly.clustering.server">
            <local-cache name="default">
                <transaction mode="BATCH"/>
            </local-cache>
        </cache-container>
        <cache-container name="web" default-cache="passivation" module="org.wildfly.clustering.web.infinispan">
            <local-cache name="passivation">
                    <redis-store xmlns="urn:infinispan:config:store:redis:8.0"
                        topology="server" socket-timeout="10000" connection-timeout="10000">
                    <redis-server host="server1" />
                    <connection-pool min-idle="6" max-idle="10" max-total="20" min-evictable-idle-time="30000" time-between-eviction-runs="30000" />
            </redis-store>
        </persistence>
            </local-cache>
        </cache-container>
        <cache-container name="ejb" aliases="sfsb" default-cache="passivation" module="org.wildfly.clustering.ejb.infinispan">
            <local-cache name="passivation">
                <locking isolation="REPEATABLE_READ"/>
                <transaction mode="BATCH"/>
                <file-store passivation="true" purge="false"/>
            </local-cache>
        </cache-container>
        <cache-container name="hibernate" module="org.infinispan.hibernate-cache">
            <local-cache name="entity">
                <transaction mode="NON_XA"/>
                <object-memory size="10000"/>
                <expiration max-idle="100000"/>
            </local-cache>
            <local-cache name="local-query">
                <object-memory size="10000"/>
                <expiration max-idle="100000"/>
            </local-cache>
            <local-cache name="timestamps"/>
        </cache-container>
    </subsystem>

But when I start Keycloak, I get this error:

'persistence' isn't an allowed element here.

Question: Is there a straightforward way to configure Keycloak to save cached data in Redis or another persistent data store?

Answer 1

The subsystem version urn:jboss:domain:infinispan:6.0 doesn't know about this schema of your xml, so you would have to either update the subsystem or if you using the latest image of Keycloak (6.0.1) maybe would be easier to just implement a new InfinispanConnectionProviderFactory, which just involves basically doing this with Wildfly:

/subsystem=keycloak-server/spi=connectionsInfinispan/:remove()
/subsystem=keycloak-server/spi=connectionsInfinispan/:add(default-provider=custom)
/subsystem=keycloak-server/spi=connectionsInfinispan/provider=custom/:add(properties={},enabled=true)

For that, of course, you would have to implement an extension and deploy it. But then at the code level, you can use the full power of the latest Infinispan.

I see that you want to use Redis, that is another big problem with it, please read this answer https://stackoverflow.com/a/57362238/571689 where I tell you the following problems that you will bump into.