Search code examples
nginxopensslssl-certificatemicroservicesself-signed-certificate

Trust Self-signed SSL/TLS Certificate for Secure Inter-service Communication

We have an orchestration of microservices running on a server. An nginx service is acting as a proxy between microservices. We would like to have all the communications on SSL with our self-signed certificates.

We want to add our private CA to every service (running on Debian Buster), so that it is considered valid everywhere within that service. We generate our server certificate and CA as follows:

# Generate Root CA Certificate
openssl genrsa -des3 -out CA-key.pem 2048
openssl req -new -key CA-key.pem -x509 -days 1000 -out CA-cert.pem

# Generate a Signing a Server Certificate
openssl genrsa -des3 -out server-key.pem 2048
openssl req –new –config openssl.cnf –key server-key.pem –out signingReq.csr
openssl x509 -req -days 365 -in signingReq.csr -CA CA-cert.pem -CAkey CA-key.pem -CAcreateserial -out server-cert.pem

However, we can't make the microservices to consider the certificate as valid and trust it. When a get request is issued using the request library of Python in the micro-service, the following exception is thrown:

requests.exceptions.SSLError: HTTPSConnectionPool(host='server.name', port=443): Max 
retries exceeded with url: /url/to/microservice2/routed/via/nginx/ (Caused by 
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate 
verify failed: self signed certificate (_ssl.c:1076)')))

What we have tried so far:

  • Copying the certificate to /usr/share/ca-certificate/ and running the sudo dpkg-reconfigure ca-certificates and/or update-ca-certificates commands.
  • Set the REQUESTS_CA_BUNDLE env variable to /path/to/internal-CA-cert.pem
  • Set the SSL_CERT_FILE env variable to /path/to/internal-CA-cert.pem

The only workaround that works is setting the valid=False in requests.get(url, params=params, verify=False, **kwargs), to ignore the validity of the SSL certificate is ignored. But, this is not the worfklow we would want to implement for all the microservices and communications.

Solution

  • The solution was to copy the self-signed server certificate (signed with our own CA) to the /usr/local/share/ca-certificates directory and use the update-ca-certificates which is shipped in debian distributions (similar solution is available for other linux distributions).

    cp /path/to/certificate/mycert.crt /usr/local/share/ca-certificates/mycert.crt
update-ca-certificates

    However, the tricky part is that the above solution is not sufficient for the python request library to consider the certificate as valid. To resolve that, one has to append the self-signed server certificate to the cat-certificates.crt and then set the environment variable REQUESTS_CA_BUNDLE to that appended file. 

    cat /path/to/certificate/mycert.crt >>/etc/ssl/certs/ca-certificates.crt
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt