I've got a question about the repository pattern. I'm working on a distributed project where the data is kept in memory but there is a database for durability of the data. To scale the system the data is sharded across multiple instances of the app. For this reason, every time there is a problem with any of the nodes, the data is reloaded from database and resharded using a distributed queue.
We created a repository interface so the domain talks to it to deal with data. The implementation of this interface uses a map as internal cache and Hibernate to talk to the database.
Because the cache has to be cleared whenever an instance of the application is created or deleted to start resharding, I have exposed a method in the repo to this aim. Is this the right way to do it? Isn't this forcing the interface to expose a method related to the internals of the implementation? Another solution would be to inject the cache into the service responsible to trigger the refresh but that would mean leaking information about the persistence layer implementation.
Any ideas?
Thanks.
EDIT
A little more about the app. The reason why the sharding was put in place was to be able to scale the app in case of data growing. It needs to subscribe to different channels that send messages up to 2 per second while these channels can be up to several thousand. For every message received, it needs to go through all the data, find the ones that match the info in the message and then send a notification for every one of them. To be able to scale this, the data is sharded across instances, at the moment two, so each instance takes care of the data it possesses.
The data is created by users, there are some REST endpoints created for that. In that sense it behaves as a simple app where a client can create, update, delete and get the data it has created. The repository exposes the typical CRUD operations and methods to query based on specific info and the implementation takes care of adding the data to the cache and to the database and fetch from one or the other depending on the request. There is some more complexity in the app as, in case of a cache miss, we cannot simply fetch the data from the database and put it in the cache as what this implies is that, if the data is not there, there must be another instance in the cluster that contains the data in its own cache. To handle this, we have queues and topics we use to redirect requests.
For this reason we need to clear the cache. Every time there is a change in the topology of the cluster, it means either that we have to rebalance the data in the cache in case there is a new node or that we need to get again the data from database as one of the caches has gone down. As we don't know exactly what data is missing, we clear the caches and repopulate them from database, using one node as a data loader and a distributed queue to shard the data. This job is done pretty fast so we just reject any request from the client during this time. The app doesn't have that many concurrent requests from clients, the problem is with the subscription channels with which we have to keep up.
Indeed you don't want to have a clear() method exposed.
Let's say you have two services. The repository as you have described, and a cluster manager, which knows about the cluster state and when it changes.
Now you can let the cluster manager consume the repository API and add code to call all the needed operation when there is a topology change. But, there is only a dependency from the repository implementation to the cluster manager, not the other way around. The cluster manager, doesn't need to know the repository to provide its service to other clients.
The only problem is, that the calling is reversed, the cluster manager needs to call the repository. To solve this, define an event (e.g. TopologyChanged), that is part of the cluster manager API and let the repository implementation tell the cluster manager, that it wants to get notified.
Side note:
There are products available which already do exactly this kind of operations, e.g. Apache Ignite, Hazelcast, Infinispan.