Search code examples
pythondaskbcolz

saving dask dataframe in bcolz format


The dask documentation states: "BColz is an on-disk, chunked, compressed, column-store. These attributes make it very attractive for dask.dataframe which can operate particularly well on it. There is a special from_bcolz function."

However, I could not find an example how to save a dask dataframe to bcolz. What is the recommended way to do this?


Solution

  • I created a pull request to implement this. Until it is merged into the master branch you can find it here:

    https://github.com/dask/dask/pull/1386

    If you don't want to edit your own Dask implementation you can just copy the to_bcolz method.