Python UrlLib3 - Can't download file due to SSL Error even when ssl verification is disabled

Question

I am unable to download a file using this piece of code:

import requests

response = requests.get('https://download.inep.gov.br/informacoes_estatisticas/indicadores_educacionais/taxa_transicao/tx_transicao_municipios_2019_2020.zip', stream=True, verify=False)
with open('tx_transicao_municipios_2019_2020.zip', 'wb') as f:
    for chunk in response.iter_content(chunk_size=1024): 
        if chunk: 
            f.write(chunk)

I keep getting this error even when verify=False is setted:

urllib3.exceptions.SSLError: [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)

When using Chrome, I am able to download the file.

Using verify=certifi.where() doesn't work also.

Environment

• Windows 10 Enterprise 22H2 (19045.3448);
• Python v3.11.5;
• OpenSSL v3.0.9;
• Urllib3 v2.0.6;
• Requests v2.31.0;
• Certifi v2023.7.22;

Also tried in MacOS Catalina (10.15) and MacOS Big Sur (11.x) with no success.

What am I doing wrong here?

Answer 1

Try:

import ssl
import warnings

import requests
import requests.packages.urllib3.exceptions as urllib3_exceptions

warnings.simplefilter("ignore", urllib3_exceptions.InsecureRequestWarning)


class TLSAdapter(requests.adapters.HTTPAdapter):
    def init_poolmanager(self, *args, **kwargs):
        ctx = ssl.create_default_context()
        ctx.check_hostname = False
        ctx.set_ciphers("DEFAULT@SECLEVEL=1")
        ctx.options |= 0x4
        kwargs["ssl_context"] = ctx
        return super(TLSAdapter, self).init_poolmanager(*args, **kwargs)


url = "https://download.inep.gov.br/informacoes_estatisticas/indicadores_educacionais/taxa_transicao/tx_transicao_municipios_2019_2020.zip"

with requests.session() as s:
    s.mount("https://", TLSAdapter())

    response = s.get(url, verify=False, stream=True)

    with open("tx_transicao_municipios_2019_2020.zip", "wb") as f:
        for chunk in response.iter_content(chunk_size=1024):
            if chunk:
                f.write(chunk)

Downloads tx_transicao_municipios_2019_2020.zip:

-rw-r--r-- 1 root root 17298416 okt 16 20:04 tx_transicao_municipios_2019_2020.zip