Search code examples
pythonanywhere

SSL: CERTIFICATE_VERIFY_FAILED trying to scrap via ssl on PythonAnywhere using python


While trying to get an external JSON file using python on PythonAnywhere

import http.client
conn = http.client.HTTPSConnection("www.tesourodireto.com.br")
conn.request("GET", "/json/br/com/b3/tesourodireto/service/api/treasurybondsinfo.json")

I got CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

I know that I need to import the certificate chain used by this website. I found this bash script to get it:

openssl s_client -showcerts -connect www.tesourodireto.com.br:443 </dev/null 2>/dev/null|openssl x509 -outform PEM >mycertfile.pem

Now I need to import it in the trusted cas on the os:

import certifi
certifi.where()
'/usr/local/lib/python3.10/site-packages/certifi/cacert.pem'

But it requires me root permissions:

cat mycertfile.pem > /usr/local/lib/python3.10/site-packages/certifi/cacert.pem
bash: /usr/local/lib/python3.10/site-packages/certifi/cacert.pem: Permission denied

Sudo command is out of question:

sudo su -
bash: sudo: command not found

How can I accomplish this on PythonAnywhere, being a normal user? I don't want to disable SSL chain verification...


Solution

  • If you have a complete cert file, you can pass it directly to requests:

    requests.get('https://example.com', verify='/path/to/certfile')