Search code examples
azurecachingredisservicestackservicestack.redis

ServiceStack.Redis timeout on Azure

I'm moving my ServiceStack API from Linux/Mono (On my own hardware) to the Azure App Service, using SS 4.5.2. My Redis cache is 3.2 running on a Linux VM. I am NOT using the Azure Redis service.

I'm seeing this exception being thrown seemingly at random:

RedisException - Exceeded timeout of 00:00:03

The exception seems to be thrown from services using the authenticate attribute, as every stack trace includes "ServiceStack.ServiceExtensions.GetSession" and "ServiceStack.AuthenticateAttribute.Execute". I'm currently only using Redis for session storage, so this bit is no surprise.

I'm registering my ICacheClient as follows, which has been working on my previous Linux/Mono setup in production for some time:

container.Register<IRedisClientsManager>(c =>
    new RedisManagerPool("SomeRedisMachine:6379")); 

container.Register(c => c.Resolve<IRedisClientsManager>().GetCacheClient());

I did see the post on the SS forms: https://forums.servicestack.net/t/redis-exception-exceeded-timeout-of-00-00-03/2301 - However this only seems to apply when using the Azure Redis service, which I am not.

Given this info, there are several questions I will have to find an answer to:

  • Does moving the Redis VM to a higher tier have any positive impact on this? Currently the VM is one of the smallest available. I will try this today. Update: Increasing VM size doesn't seem to resolve the issue.
  • Is the latency between Redis VM and the App Service too great? I can't imagine this is a problem, given that the are within the same datacenter.
  • Is there some subtle difference between Linux/Mono and Azure App Service (Windows, Presumably backed by some sort of modified IIS) which I need to take into account?

Thanks in advance for any insight!

Update

As per mythz suggestion, I did update to SS v4.5.4. The exception is still thrown, however with a slightly different message (Exceeded timeout of 00:00:10) consistent with the mentioned updated timeout.

I decided to target an Azure Redis Service (As opposed to a Linux VM). I so far see no errors related to Redis in either ServiceStack 4.5.2 or 4.5.4. Perhaps whatever hosts Microsoft is using for their own Redis service have a better network connection and/or are closer to the cluster handling app services.

Solution

  • The Redis TimeoutException is a symptom of an unhealthy environment which is preventing the Redis Client from establishing a TCP connection. This could be as a result of your redis-server instance or Network being overloaded or unreliable.

    You can increase the Timeout to give the Redis Client more time to make a connection with:

    RedisConfig.DefaultRetryTimeout = 10 * 1000;

    Which is also the new default timeout from ServiceStack.Redis v4.5.4+.