Search code examples
python-requeststravis-cicertifi

SSL Certificate Failure on Travis, but works fine locally (using Requests and Certifi)

I am scraping the following site using requests and certifi: https://ecf.ared.uscourts.gov/

When I scrape it on my local machine, it works fine, but when I run automated tests in Travis, it fails with:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

The code I'm using for this is pretty straightforward:

url = 'https://ecf.ared.uscourts.gov/cgi-bin/login.pl'
r = requests.post(
    url,
    verify=certifi.where(),
    timeout=60,
    files={
        'login': ('', username),
        'key': ('', password)
    },
)

Any ideas what's different between Travis and my local machine that would cause a certificate failure? I thought the idea of certifi was to standardize all this. (I've verified I have updated versions on both computers.)

Solution

  • Got it! Travis uses an old version of OpenSSL which doesn't support cross-signed certificates, but my local dev machine uses a modern version of OpenSSL.

    The solution here is to use certifi.old_where() which provides older, less secure certificate support. I've resigned myself to this until I can upgrade the version of OpenSSL that we have.