Search code examples
pythonapipython-requestsssl-certificatehttpx

SSL: CERTIFICATE_VERIFY_FAILED- Proxy Redirect - Company Network - Cacert Pem File Not Working - httpx/requests Package

Hello StackOverflow Community,

I know the [SSL: CERTIFICATE_VERIFY_FAILED] error is a wildly discussed issue. However, after testing different methods such as here, here, and here, I am still not able to solve this dreadful issue.

Background:

While on the company network, I try connecting to a REST API endpoint https://demo.vizionapi.com/carriers. However, the [SSL: CERTIFICATE_VERIFY_FAILED] error randomly appears.

Solution Tested:

I followed this article, passed https://demo.vizionapi.com/carriers to chrome to get all certificates( the root one, intermediate one, and website one), then installed certifi package and appended three certificates to the end of cacert.pem file such as following.

... other certificates...

-----BEGIN CERTIFICATE-----
xxxx (root)
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
xxxx (intermediate)
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
xxxx (website)
-----END CERTIFICATE-----

Then I passed the following code to build a connection with API.

import httpx
import certifi
head = {
    'X-API-Key': 'akkakakka'
}

url_end = 'https://prod.vizionapi.com/carriers' 

response = httpx.get(url=url_end,
                     verify=certifi.where(),
                     headers=head,
                     follow_redirects=True)

I pass follow_redirects as True because if this parameter is set to False, I will randomly get the 307 Proxy Redirect status code. However, when the redirect happened, I got the same [SSL: CERTIFICATE_VERIFY_FAILED] error; but when the redirect did not happen, the code worked flawlessly.

Further Investigation:

I dug a bit further and discovered that redirect happens due to our company's proxy hijacking the connection and further changing the URL.

URL when redirect not happen: https://demo.vizionapi.com/carriers

URL when the redirect happens (Pseudo): https://proxy02.xan.klklk/BNKS09NKSISKIO0987/11.11.11.111/https://demo.vizionapi.com/carriers

I tried to pass the redirected URL to Chrome and get the new certificates, but I still got the same error.

Questions:

I guess my question is how I can deal with the [SSL: CERTIFICATE_VERIFY_FAILED] error when such error is caused by the company proxy changing the URL and forcing the proxy URL added in front of the REST API URL?

Thank you for your suggestion, and any comment is welcomed.

Solution

  • You don't have all of the CA certs needed to verify whichever server you are sending requests to. Ask to get the complete CA bundle to use for verification.