Search code examples
pythonpyarrowfeather

Can you use read_feather without pyarrow package?


I need to load a feather file in python, preferably with pandas, so I tried to use read_feather, but came across the error

/pandas/io/feather_format.py", line 21, in _try_import
    raise ImportError("pyarrow is not installed\n\n"

And indeed, the codebase I am working in does not have pyarrow integrated. When reading https://pandas.pydata.org/docs/reference/api/pandas.read_feather.html, it appears pyarrow is needed for certain dtype backends, but I am using the default settings so I am not sure why it is needed.


Solution

  • We cannot read_feather without pyarrow package because feather IO functionality that is used in pandas is implemented in pyarrow, and thus pandas needs that library to be able to read or write feather files.

    from pyarrow import feather
    
    pa_table = feather.read_table(
                handles.handle, columns=columns, use_threads=bool(use_threads)
            )
    

    Even for dtype_backend == "numpy_nullable"

    if dtype_backend == "numpy_nullable":
                from pandas.io._util import _arrow_dtype_mapping
    
                return pa_table.to_pandas(types_mapper=_arrow_dtype_mapping().get
    

    Link to source code: https://github.com/pandas-dev/pandas/blob/v2.1.3/pandas/io/feather_format.py#L72-L148