Connect to an Azure hosted ElasticSearch endpoint through a SSL offloading Application Gateway

Question

I have used the template Elasticsearch Azure Marketplace to deploy an ElasticSearch cluster in Azure.

I configured it with SSL/TLS for communication with Elasticsearch via the HTTP layer through Application Gateway and everything works fine, I can log in to Kibana and see the status of my cluster nodes.

The problem is I can't connect to ElasticSearch using NEST.NET through the Azure Application Gateway from a client, it requires the certificate and password I provided when submitting the template but when I set it I get "Unable to read data from the transport connection" and "The SSL connection could not be established, see inner exception." when I send a request.

Here's the code I use from my client to connect to ElasticSearch:

    public IElasticClient Client { get; }

    public ElasticService(IConfiguration configuration)
    {
        var settings = new ConnectionSettings(new Uri(configuration["Elastic:Endpoint"]))
            .DefaultIndex("impression");
        settings.ClientCertificate(new X509Certificate2(@"C:\git\server.p12", "PASSWORD", X509KeyStorageFlags.Exportable));
        Client = new ElasticClient(settings);
    }

Answer 1

The certificate in this case isn't used for certificate authentication to Elasticsearch, as the ClientCertificate method is used for, but is used for Transport Layer Security (TLS).

A TLS certificate can be set with ServerCertificateValidationCallback

var pool = new SingleNodeConnectionPool(new Uri(configuration["Elastic:Endpoint"]));

var settings = new ConnectionSettings(pool)
    .DefaultIndex(defaultIndex)
    .BasicAuthentication("elastic", "<password>")
    .ServerCertificateValidationCallback(
        CertificateValidations.AuthorityPartOfChain(
            new X509Certificate2(@"C:\git\server.p12", "PASSWORD"))
    );

var client = new ElasticClient(settings);

Depending on how the certificate passed to Application Gateway has been generated, CertificateValidations offers AuthorityPartOfChain and AuthorityIsRoot, depending on if the certificate with which the client configured is part of the chain, or is the root certificate, respectively.