Search code examples
pythongithub-actionsshapefilegeopandasfiona

Fiona not seeing .shp file as a recognised format


All of my unittests pass on my local machine, however when I try to use a .yml file to test them every time a pull request is created, there are several failures. An example of one of the error messages is shown below:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   fiona._err.CPLE_OpenFailedError: 'static_data/england_wa_2011_clipped.shp' not recognized as a supported file format.

fiona/_err.pyx:291: CPLE_OpenFailedError

My Linux .yml file is below, I have already tried changing around the working directory and it appears to be correct. The file is not corrupted and as it is the same on both VM's I think it is an issue with Fiona. This file also has a corresponding file for testing on a Windows VM however they are spitting out the same error messages and failing the same tests.

name: Python Linux application

on:
  pull_request:
    branches: [ '**' ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python 3.8
        uses: actions/setup-python@v2
        with:
          python-version: 3.8
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          sudo apt-get install libproj-dev proj-data proj-bin
          sudo apt-get install libgeos-dev
          sudo apt-add-repository ppa:ubuntugis/ubuntugis-unstable
          sudo apt-get update
          sudo apt-get install gdal-bin libgdal-dev
          pip install GDAL==3.2.3
          pip install flake8 pytest Cython numpy pyproj pygeos
          if [ -f requirements-linux.txt ]; then pip install -r requirements-linux.txt; fi
      - name: Lint with flake8
        run: |
          # stop the build if there are Python syntax errors or undefined names
          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
      - name: Test with pytest
        run: |
          pytest

GitHub Repo: https://github.com/Zach10a/seedpod_ground_risk The branch is CI.


Solution

  • Your problem is that the actions/checkout@v2 action does not, by default, check out files stored using LFS. So while there is file named, for example, static_data/england_wa_2011_clipped.shp in your repository, the contents are going to look something like this:

    version https://git-lfs.github.com/spec/v1
    oid sha256:c60f74e3b8ed753d771378f0b03b7c8e8a84406f413a37f9f5242ac9235a2e6c
    size 114084720
    

    So Fiona is giving you an accurate error:

    E   fiona._err.CPLE_OpenFailedError: 'static_data/england_wa_2011_clipped.shp' not recognized as a supported file format.
    

    You need to instruct the checkout action to download files stored in LFS:

        steps:
          - uses: actions/checkout@v2
            with:
              lfs: true