C# LazyCache concurrent dictionary garbage collection

Question

Been having some problems with a web based .Net(C#) application. I'm using the LazyCache library to cache frequent JSON responses (some in & around 80+KB) for users belonging to the same company across user sessions.

One of the things we need to do is to keep track of the cache keys for a particular company so when any user in the company makes mutating changes to items being cached we need to clear the cache for those items for that particular company's users to force the cache to be repopulated immediately upon the receiving the next request.

We choose LazyCache library as we wanted to do this in memory without needing to use an external cache source such as Redis etc as we don't have heavy usage.

One of the problems we have using this approach is we need to keep track of all the cache keys belonging to a particular customer anytime we cache. So when any mutating change is made by company user's to the relevant resource we need to expire all the cache keys belonging to that company.

To achieve this we have a global cache which all web controllers have access to.

private readonly IAppCache _cache = new CachingService();

protected IAppCache GetCache()
{
    return _cache;
}

A simplified example (forgive any typos!) of our controllers which use this cache would be something like below

[HttpGet]
[Route("{customerId}/accounts/users")]
public async Task<Users> GetUsers([Required]string customerId)
{
    var usersBusinessLogic = await _provider.GetUsersBusinessLogic(customerId)

    var newCacheKey= "GetUsers." + customerId;

    CacheUtil.StoreCacheKey(customerId,newCacheKey)

    return await GetCache().GetOrAddAsync(newCacheKey, () => usersBusinessLogic.GetUsers(), DateTimeOffset.Now.AddMinutes(10));
}

We use a util class with static methods and a static concurrent dictionary to store the cache keys - each company (GUID) can have many cache keys.

private static readonly ConcurrentDictionary<Guid, ConcurrentHashSet<string>> cacheKeys = new ConcurrentDictionary<Guid, ConcurrentHashSet<string>>();

public static void StoreCacheKey(Guid customerId, string newCacheKey)
{
    cacheKeys.AddOrUpdate(customerId, new ConcurrentHashSet<string>() { newCacheKey }, (key, existingCacheKeys) =>
    {
        existingCacheKeys.Add(newCacheKey);
        return existingCacheKeys;
    });
}

Within that same util class when we need to remove all cache keys for a particular company we have a method similar to below (which is caused when mutating changes are made in other controllers)

public static void ClearCustomerCache(IAppCache cache, Guid customerId)
{
    var customerCacheKeys = new ConcurrentHashSet<string>();

    if (!cacheKeys.TryGetValue(customerId,out customerCacheKeys))
    {
        return new ConcurrentHashSet<string>();
    }


    foreach (var cacheKey in customerCacheKeys)
    {
        cache.Remove(cacheKey);
    }

    cacheKeys.TryRemove(customerId, out _);
}

We have recently been getting performance problems that our web requests response time slow significantly over time - we don't see significant change in terms of the number of requests per second.

Looking at the garbage collection metrics we seem to notice a large Gen 2 heap size and a large object size which seem to going upwards - we don't see memory been reclaimed.

We are still in the middle of debugging this but I'm wondering could using the approach described above lead to the problems we are seeing. We want thread safety but could there be an issue using the concurrent dictionary we have above that even after we remove items that memory is not being freed leading to excessive Gen 2 collection.

Also we are using workstation garbage collection mode, imagine switching to server mode GC will help us (our IIS server has 8 processors + 16 GBs ram) but not sure switching will fix all the problems.

Answer 1

Large objects (> 85k) belong in gen 2 Large Object Heap (LOH), and they are pinned in memory.

1. GC scans LOH and marks dead objects
2. Adjacent dead objects are combined into free memory
3. The LOH is not compacted
4. Further allocations only try to fill in the holes left by dead objects.

No compaction, but only reallocation may lead to memory fragmentation. Long running server processes can be done in by this - it is not uncommon. You are probably seeing fragmentation occur over time.

Server GC just happens to be multi-threaded - I wouldn't expect it to solve fragmentation.

You could try breaking up your large objects - this might not be feasible for your application.

You can try setting LargeObjectHeapCompaction after a cache clear - assuming it's infrequent.

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect();

Ultimately, I'd suggest profiling the heap to find out what works.