regarding of improvement of the efficiency of a cache heavy system

Question

i'm about to improve the efficiency of a cache heavy system, which has the following properties/architecture:

The system has 2 components, a single instance backend and multiple frontend instances, spread across remote data centers.

The backend generates data and writes it to a relational database that is replicated to multiple data centers.

The frontends handle client requests (common web traffic based) by reading data from the database and serving it. Data is stored in a local cache for an hour before it expires and has to be retrieved again.

(The cache’s eviction policy is LRU based).

I want to mention that there are two issues with the implementation above:

It turns out that many of the database accesses are redundant because the underlying data didn't actually change. On the other hand, a change isn't reflected until the cache TTL elapses, causing staleness issues.

can you advice for a solution that fixes both of these problems?

should the solution change if the data was stored in nosql db like cassandra and not a classic database?

Answer 1

Unfortunately, there is no silver bullet here. There are two obvious variants:

1. Keep long TTL or cache forever, but invalidate the cache data when it is updated. This might get quite complex and error prone
2. Simply lower the TTL to get faster updates. The low TTL approach is IMHO the KISS approach. We go as low as 27 seconds. A cache with this low TTL has not a lot hits during normal operation, but helps a lot when a flash crowd hits your application

In case your database is powerful enough and has acceptable latency the approach 2 is the simplest one.

If your database, does not have acceptable latency, or maybe your application is doing a multiple of sequential reads from the database per web request, then your can use a cache that provides refresh ahead or background refresh. This means, the cache refreshes the entries automatically and there is no additional latency except for the first read. However, this approach come with the downside of increasing the database load.

Cassandra may not support the same access strategies like the classic database. Changing to Cassandara will affect your caching as well, e.g. in case you cache also query results. However, the high level concept keeps the same. Your data access layers may change to an asynchronous or reactive pattern, since Cassandara has support for that.

If you want to do invalidation (solution 1), with Cassandara, you can get information from the database which data has updated, see CASSANDRA-8844. You may get similar information from "classical" SQL databases, but that is a vendor specific feature.