Search code examples
pythonnetwork-programmingdnspython-requests

How to make `requests` use a different hostname for TLS validation than DNS resolution?

There's a service running in a domain such as vpc-elasti-xyz.us-east-1.es.amazonaws.com. It listens for regular HTTPS connections and has a valid certificate for that domain.

I need a Python app using the requests module to connect to this service securely, but through a different hostname -- my-search-domain.example.com for example.

I'm wondering if there's a way to do something like this:

s = Session()
req = Request('GET', 'my-search-domain.example.com')
prepped = s.prepare_request(req)

# ?????
prepped.expected_host_name = 'vpc-elasti-xyz.us-east-1.es.amazonaws.com'

resp = s.send(prepped)

Possible solutions I thought about:

  1. Adding that host name to the original service's certificates -- unfortunately not possible.
  2. Putting a reverse proxy in front of the service and do TSL termination -- doable, but I'd like to avoid an extra moving piece if possible.
  3. Using a CNAME record for my-search-domain.example.com that points to vpc-elasti-xyz.us-east-1.es.amazonaws.com. This wouldn't work OOB since the domain name doesn't match, but a workaround would be to retrieve the domain name from DNS in my code and just use that to connect directly. That's what I have right now and it works, but I'd like to not do that "manual DNS query" if possible. Also, at this point DNS is just a convenient data store -- no different from a configuration file or anything else.
Solution

  • Requests supports SSL Verification by default. However, it relies on the user making a request with the URL that has the hostname in it. If, however, the user needs to make a request with the IP address, they cannot actually verify a certificate against the hostname they want to request.

    To accomodate this need, there is HostHeaderSSLAdapter. Example usage: 

    import requests
from requests_toolbelt.adapters import host_header_ssl
s = requests.Session()
s.mount('https://', host_header_ssl.HostHeaderSSLAdapter())
s.get("https://my-search-domain.example.com", headers={"Host": "vpc-elasti-xyz.us-east-1.es.amazonaws.com"})

    https://toolbelt.readthedocs.io/en/latest/adapters.html#hostheaderssladapter