Fiona Driver Error when downloading files via URL

Question

This is simple to test if you get the error on your side:

import geopandas as gpd
gdf = gpd.read_file('https://hepgis.fhwa.dot.gov/fhwagis/AltFuels_Rounds1-5_2021-05-25.zip')

File "fiona/ogrext.pyx", line 540, in fiona.ogrext.Session.start
File "fiona/_shim.pyx", line 90, in fiona._shim.gdal_open_vector
fiona.errors.DriverError: '/vsimem/6101ab5f23764c15b5fe47aa52a049d6' not recognized as a supported file format.

Interestingly, I have received this error for other URLs recently and thought there was something wrong with the URL. But, now I suspect that something else is going on since it is happening with more than one URL. On the other hand, some URLs don't have this issue. One other interesting thing, this error only occurs sometimes. For instance, if I rerun that command it will work maybe 1 out of 20 times.

My Fiona version: fiona 1.8.20 py39hea8b339_1 conda-forge

Any help would be much appreciated.

Answer 1

Investigating, the URL does not return a zip file. See code below, it actually returns a HTML input page...

import geopandas as gpd
import requests, io
from pathlib import Path
from zipfile import ZipFile, BadZipFile
import urllib
import fiona

url = "https://hepgis.fhwa.dot.gov/fhwagis/AltFuels_Rounds1-5_2021-05-25.zip"

try:
    gdf = gpd.read_file(url)
except Exception:
    f = Path.cwd().joinpath(urllib.parse.urlparse(url).path.split("/")[-1])

    r = requests.get(url, stream=True, headers={"User-Agent": "XY"})
    with open(f, "wb") as fd:
        for chunk in r.iter_content(chunk_size=128):
            fd.write(chunk)
    try:
        zfile = ZipFile(f)
        zfile.extractall(f.stem)
    except BadZipFile:
        with open(f) as fh:
            print(fh.read())