Search code examples
python-3.xproxypython-requestsdnsrhel

Python requests using a proxy returns 'service not known, socket.getaddrinfo, socket.gaierror: [Errno -2] on a valid url


Error:

Failed to establish a new connection: [Errno -2] Name or service not known File "///****/lib64/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "a", line 745, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known

This problem is only happening on Redhat 7 in production. I have the exact same code running on ubuntu from a different network and I have no problems. Everything works perfect.

My code calls 3 API's. 2 do not work on redhat but one does. I am using an anonymous proxy. The other strange thing is I can telnet to all theses api's on redhat through the same proxy and they work fine. But just not through requests.

UPDATE: the server is on a corporate network which does not resolve dns. The dns is resolved on the external proxy

To me it seems as though the requests library is not communicating with an updated dns resolver or something like this, but I am a bit hazy on dns to be honest. I don't think there is any point in posting the code as I said it all works perfect on ubuntu. Just for the sake of It I will post one of the calls. I am using ... requests==2.25.1 Python 3.6.12 Redhat 7

        HTTP_PROXIES={'http': 'http://xx.xx.xx.xx:8080'}
        headers = {'accept': 'application/json', 'Content-Type': 'application/json',}
        data = json.dumps({"key":list_values})

        response = requests.post('https://someapiaddress.com/', 
        proxies=HTTP_PROXIES, headers=headers, data=data)

Any help , very much appreciated


Solution

  • The solution was very simple in the end. The reason one api worked and the other didn't work was because the API that worked was non secure http and the ones that didn't work were secure: https Solution was. instead of...

    HTTP_PROXIES={'http': 'http://xx.xx.xx.xx:8080'}

    It should have been

     HTTP_PROXIES={'http': 'http://xx.xx.xx.xx:8080','https': 'https://xx.xx.xx.xx:8080'}