Search code examples
c#redisstackexchange.redisamazon-elasticache

Error trying to connect to AWS Elasticache with StackExchange.Redis


I'm using AWS Elasticache for Redis. It's configured with cluster mode enabled. The engine version is 6.2.6.

The code samples below run from the exact same machine with all the permissions, certificates and route/network/acl rules needed.

Let me start with the following small snippet of python code:

fromt redis.cluster import RedisCluster
rc = RedisCluster(host="myhost.mydomain.internal", ssl=True, passowrd="mysupersecretpassword")
rc.ping()

It works and returns True.

Below the csharp code that I'm trying to use to do almost the same thing:

using StackExchange.Redis;
using System.Net;
using System.Threading.Tasks;

namespace CSharpRedisCli
{
    class Program
    {
        static readonly ConnectionMultiplexer redis = ConnectionMultiplexer.Connect(
            new ConfigurationOptions{
                EndPoints = { new DnsEndPoint("myhost.mydomain.internal", 6379) },
                ResolveDns = true,
                Ssl = true,
                Password = "mysupersecretpassword",
                AbortOnConnectFail=false
            });
        
        static async Task Main(string[] args)
        {
            var db = redis.GetDatabase();
            var pong = await db.PingAsync();
            Console.WriteLine(pong);
        }
    }
}

But than i run in the following issue:

Unhandled exception. StackExchange.Redis.RedisConnectionException: The message timed out in the backlog attempting to send because no connection became available - Last Connection Exception: AuthenticationFailure on myhost.mydomain.internal:6379/Interactive, Initializing/NotStarted, last: NONE, origin: ConnectedAsync, outstanding: 0, last-read: 7s ago, last-write: 7s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 7s ago, v: 2.6.104.40210, command=PING, timeout: 5000, inst: 0, qu: 0, qs: 0, aw: False, bw: CheckingForTimeout, rs: NotStarted, ws: Initializing, in: 0, last-in: 0, cur-in: 0, sync-ops: 0, async-ops: 1, serverEndpoint: myhost.mydomain.internal:6379, conn-sec: n/a, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: C02FJ1DMMD6Q(SE.Redis-v2.6.104.40210), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=0,Free=32767,Min=16,Max=32767), POOL: (Threads=6,QueuedItems=0,CompletedItems=174), v: 2.6.104.40210 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
 ---> StackExchange.Redis.RedisConnectionException: AuthenticationFailure on myhost.mydomain.internal:6379/Interactive, Initializing/NotStarted, last: NONE, origin: ConnectedAsync, outstanding: 0, last-read: 7s ago, last-write: 7s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 7s ago, v: 2.6.104.40210
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure: RemoteCertificateNameMismatch
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.AuthenticateAsClient(SslClientAuthenticationOptions sslClientAuthenticationOptions)
   at StackExchange.Redis.ExtensionMethods.AuthenticateAsClientUsingDefaultProtocols(SslStream ssl, String host) in /_/src/StackExchange.Redis/ExtensionMethods.cs:line 206
   at StackExchange.Redis.ExtensionMethods.AuthenticateAsClient(SslStream ssl, String host, Nullable`1 allowedProtocols, Boolean checkCertificateRevocation) in /_/src/StackExchange.Redis/ExtensionMethods.cs:line 196
   at StackExchange.Redis.PhysicalConnection.ConnectedAsync(Socket socket, LogProxy log, SocketManager manager) in /_/src/StackExchange.Redis/PhysicalConnection.cs:line 1500
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at CSharpRedisCli.Program.Main(String[] args) in Program.cs:line 21
   at CSharpRedisCli.Program.<Main>(String[] args)

What am I missing with my csharp code ?

EDIT

The host myhost.mydomain.internal is a route53 CNAME record pointing to Elasticache configuration endpoint.

When calling the configuration endpoint directly, it works.

There is something going on when calling the route53 host. While my python code is able to handle this call, something need to be configured on the dotnet/stackexchanghe.redis side.


Solution

  • When using an EndPoint that is different from the redis endpoint configuration, you need to set the SslHost configuration option.

    Based on that, I have written a code to query the redis endpoint from my R53 record:

    using Amazon.Route53;
    using Amazon.SecretsManager;
    using Amazon.Route53.Model;
    using StackExchange.Redis;
    using System.Net;
    using System.Net.Security;
    using System.Security.Authentication;
    using Amazon.SecretsManager.Model;
    
    namespace CSharpRedisCli
    {
        class Program
        {
    
            private static async Task<String> getRedisEndpointFromR53(String r53RecordName, String hostedZoneName)
            {
                var route53Client = new AmazonRoute53Client();
    
                var listHostedZoneResponse = await route53Client.ListHostedZonesByNameAsync(
                                new ListHostedZonesByNameRequest { DNSName = hostedZoneName });
    
                var hostedZoneId = listHostedZoneResponse.HostedZones.First().Id.Split("/").Last();
    
                var listRecordSetResponse = await route53Client.ListResourceRecordSetsAsync(
                    new ListResourceRecordSetsRequest
                    {
                        HostedZoneId = hostedZoneId,
                        StartRecordName = r53RecordName
                    });
    
                return listRecordSetResponse.ResourceRecordSets.First().ResourceRecords.First().Value;
            }
    
            private static async Task<String> getRedisPassword(String secretId)
            {
                var ssmClient = new AmazonSecretsManagerClient();
    
                var ssmResponse = await ssmClient.GetSecretValueAsync(new GetSecretValueRequest
                {
                    SecretId = secretId
                });
    
                var pwd = ssmResponse.SecretString;
    
                return pwd;
            }
    
            static void Main(string[] args)
            {
                var r53RecordName = "myhost.mydomain.internal";
                var hostedZoneName = "mydomain.internal.";
                var secretId = "mypasswordid";
    
                var configurationOptions = new ConfigurationOptions
                {
                    EndPoints = { new DnsEndPoint(r53RecordName, 6379) },
                    Ssl = true,
                    Password = getRedisPassword(secretId).Result,
                    SslProtocols = SslProtocols.Tls12 | SslProtocols.Tls13
                };
    
                configurationOptions.CertificateValidation += (sender, certificate, chain, sslPolicyErrors) =>
                {
                    if (sslPolicyErrors == SslPolicyErrors.None)
                    {
                        Console.WriteLine("No certificate error");
                        return true;
                    }
    
                    if (sslPolicyErrors == SslPolicyErrors.RemoteCertificateNameMismatch)
                    {
                        Console.WriteLine("Remote certificate name mismatch. Updating SslHost...");
                        configurationOptions.SslHost = getRedisEndpointFromR53(r53RecordName, hostedZoneName).Result;
                        return true;
                    }
    
                    Console.WriteLine("Ops");
                    return false;
                };
    
                var redis = ConnectionMultiplexer.Connect(configurationOptions);
                var db = redis.GetDatabase();
                var pong = db.Ping();
    
                Console.WriteLine("ack: " + pong);
            }
        }
    }
    

    I still need to figure out how the Python library handles this and see if it's possible to do the same with the StackExchange.Redis library.