pkg_resources DeprecationWarning for use of '..' in resource path

Question

In a Python package I have a data file which is nested 'above' a file which requires it. e.g.

package
  data
    data.csv
  utils
    util.py

In util.py I obtain the file as a stream using pkg_resources; this allows the code to be run from source and when installed as a package.

    data_stream = pkg_resources.resource_stream(
        __name__, os.path.join("..", "data", "data.csv")
    )

This works fine, and allows the data file to be retrieved as expected.

However, when I run tests which use the code in util.py, a Deprecationwarning is revealed, stating "DeprecationWarning: Use of .. or absolute path in a resource path is not allowed and will raise exceptions in a future release."

But I'm not sure what the correct way to approach this is?

Answer 1

It seems this issue did not only cause a DeprecationWarning, but led to an error when installing from the packaged .egg file (but not the .whl) - previously documented here: package data not installed from python .egg file

My workaround to both of these was to place a data_resources.py module at the same level as the data files, which defines the data structures from the on-disk resources. This avoids having to use paths with '..' in them.

E.g.

package
  data
    data_resources.py
    data.csv
  utils
    util.py

where data_resources.py may contain something like:

data_stream = pkg_resources.resource_stream(__name__, "data.csv")
data_df = pd.read_csv(data_stream)

and util.py can simply import:

from package.data.data_resources import data_df

I'm assuming this is a more correct approach to this situation, since it suppresses the DeprecationWarnings, and allows installation from .egg and .whl files.