I'm trying to import dask_cuda as the examples:
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
But I receive the following error:
ImportError Traceback (most recent call last)
Input In [3], in <cell line: 1>()
----> 1 from dask_cuda import LocalCUDACluster
File ~/miniconda3/lib/python3.8/site-packages/dask_cuda/__init__.py:5, in <module>
2 import dask.dataframe.shuffle
4 from ._version import get_versions
----> 5 from .cuda_worker import CUDAWorker
6 from .explicit_comms.dataframe.shuffle import get_rearrange_by_column_tasks_wrapper
7 from .local_cuda_cluster import LocalCUDACluster
File ~/miniconda3/lib/python3.8/site-packages/dask_cuda/cuda_worker.py:19, in <module>
14 from distributed.proctitle import (
15 enable_proctitle_on_children,
16 enable_proctitle_on_current,
17 )
18 from distributed.utils import parse_bytes
---> 19 from distributed.worker import parse_memory_limit
21 from .device_host_file import DeviceHostFile
22 from .initialize import initialize
ImportError: cannot import name 'parse_memory_limit' from 'distributed.worker' (/home/shambakey1/miniconda3/lib/python3.8/site-packages/distributed/worker.py)
I googled this error, but couldn't find anything related to it. I thought it was a DASK problem, so I re-tried with only from dask.distributed import Client
, and it worked. So, I guess this error is related to dask_cuda, but I don't know how.
The version I have are:
Regards
Your dask version is rather old, the parse_memory_limit
was added fairly recently in 2022, see this PR.
I would recommend not pinning dask and let conda
find the most suitable version. If that's not an option, then dask=2022.7.0
works for me with dask-cuda=2022.6.0
.