Search code examples
python-3.xgoogle-colaboratoryrapids

RAPIDS in Colab AttributeError: module 'cudf' has no attribute '_lib'

I already install RAPIDS in Colab with no issues until I tried to import cuml library. I have fortunaly the Tesla 4 as GPU.

This is how I installed RAPIDS

# clone RAPIDS AI rapidsai-csp-utils scripts repo
>> !git clone https://github.com/rapidsai/rapidsai-csp-utils.git    
# install RAPIDS
>> !bash rapidsai-csp-utils/colab/rapids-colab.sh
>> import sys, os
# set necessary environment variables 
>> dist_package_index = sys.path.index('/usr/local/lib/python3.6/dist-packages')
>> sys.path = sys.path[:dist_package_index] + ['/usr/local/lib/python3.6/site-packages']+sys.path[dist_package_index:]
>> sys.path    
# update pyarrow & modules 
>> exec(open('rapidsai-csp-utils/colab/update_modules.py').read(), globals())
Enjoy using RAPIDS!
RAPIDS Version to install is 0.11
Checking for GPU type:
***********************************************************************
Woo! Your instance has the right kind of GPU, a 'Tesla T4'!
***********************************************************************

************************************************
Your Google Colab instance has RAPIDS installed!
************************************************
***********************************************************************
Let us check on those pyarrow and cffi versions...
***********************************************************************
You're running pyarrow 0.15.0 and are good to go!
unloaded cffi 1.11.5
loaded cffi 1.11.5

And when i tried to import:

>> import cuml

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-a450aff8eac6> in <module>()
----> 1 import cuml

5 frames
/usr/local/lib/python3.6/site-packages/cudf/core/dataframe.py in <module>()
     22 
     23 import cudf
---> 24 import cudf._lib as libcudf
     25 import cudf._libxx as libcudfxx
     26 from cudf._libxx.null_mask import MaskState, create_null_mask

AttributeError: module 'cudf' has no attribute '_lib'

Also I put this:

>> ! conda list | grep cudf
>> ! conda list | grep cuml
>> ! conda list | grep cugraph
>> ! conda list | grep numpy
>> ! conda list | grep pandas

cudf                      0.13.0                   py36_0    rapidsai/label/main
dask-cudf                 0.13.0                   py36_0    rapidsai/label/main
libcudf                   0.13.0               cuda10.0_0    rapidsai/label/main
cuml                      0.13.0          cuda10.0_py36_0    rapidsai/label/main
libcuml                   0.13.0               cuda10.0_0    rapidsai/label/main
libcumlprims              0.13.0               cuda10.0_0    nvidia
cugraph                   0.13.0                   py36_0    rapidsai/label/main
libcugraph                0.13.0               cuda10.0_0    rapidsai/label/main
numpy                     1.17.5           py36h95a1406_0    conda-forge
pandas                    0.25.3           py36hb3f55d8_0    conda-forge

This is my issue, I'm trying wiht RAPIDS to use his T-SNE, is faster than the Scipy T-SNE.

Solution

  • Thanks for sharing this. This issue was cause by an update to numba (0.48.0 -- 0.49.0) that rendered it incompatible with cudf. It was solved by this PR https://github.com/rapidsai/rapidsai-csp-utils/pull/18 which locks numba to 0.48.0 for 0.13 and below.