can't open shape file with GeoPandas

Question

I don't seem to be able to open the zip3.zip shape file I download from (http://www.vdstech.com/usa-data.aspx)

Here is my code:

import geopandas as gpd
data = gpd.read_file("data/zip3.shp")

this gives me the error:

CPLE_AppDefinedError: b'Recode from CP437 to UTF-8 failed with the error: "Invalid argument".'

Answer 1

As per my answer on this question, seems like your dataset contains non-UTF characters. If you are facing this similar issue, chances are that using encoding-"utf-8" won't help as Fiona's open() call will still fail.

If other solutions don't work, two solutions I propose that solved this issue are:

1. Open your shapefile on a GIS editor (like QGis), then save it again making sure you select the Encoding option to "UTF-8". After this you should have no problem when calling gpd.read_file("data/zip3.shp").

2. You can also achieve this format change in Python using GDAL, by reading your shapefile and saving it again. This will effectively change the encoding to UTF-8, as this is the default encoding as indicated in the docs for the CreateDataSource() method. For this try the following code snippet:

   from osgeo import ogr
   
   driver = ogr.GetDriverByName("ESRI Shapefile")
   ds = driver.Open("nbac_2016_r2_20170707_1114.shp", 0) #open your shapefile
   #get its layer
   layer = ds.GetLayer()
   
   #create new shapefile to convert
   ds2 = driver.CreateDataSource('convertedShape.shp')
   #create a Polygon layer, as the one your Shapefile has
   layer2 = ds2.CreateLayer('', None, ogr.wkbPolygon)
   #iterate over all features of your original shapefile
   for feature in layer:
      #and create a new feature on your converted shapefile with those features
      layer2.CreateFeature(feature)
   #proper closing
   ds = layer = ds2 = layer2 = None