Search code examples
pythonanacondacondaxgboostkaggle

```pipdeptree --reverse --packages xgboost``` - how to resovle library incompatibility


GOAL

  • My final goal would be to run this kaggle project locally in a conda environment.
  • I am running this on Ubuntu 20.04 LTS, conda environment, python 3.8

INPUT

  • I have got recommended to run this code because I have library dependencies issues.
pipdeptree --reverse --packages xgboost

OUTPUT

Warning!!! Possibly conflicting dependencies found:
* wandb==0.10.4
 - subprocess32 [required: >=3.5.3, installed: ?]
* spyder==4.1.5
 - pyqt5 [required: <5.13, installed: ?]
 - pyqtwebengine [required: <5.13, installed: ?]
* QDarkStyle==2.8.1
 - helpdev [required: >=0.6.10, installed: ?]
* Flask-Compress==1.5.0
 - brotli [required: Any, installed: ?]
* dask-xgboost==0.1.11
 - xgboost [required: <=0.90, installed: 1.2.0]
* dash==1.16.3
 - dash-core-components [required: ==1.12.1, installed: 1.3.1]
 - dash-html-components [required: ==1.1.1, installed: 1.0.1]
 - dash-renderer [required: ==1.8.2, installed: 1.1.2]
 - dash-table [required: ==4.10.1, installed: 4.4.1]
------------------------------------------------------------------------
xgboost==1.2.0
  - dask-xgboost==0.1.11 [requires: xgboost<=0.90]

UPDATE

Now I have reinstalled XGBoost and

conda install -c conda-forge xgboost

when I run my code's import in jupyter it says

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-f5a95d2c9926> in <module>
      3 from dask.distributed import Client
      4 from dask import array as da
----> 5 import xgboost as xgb
      6 from xgboost import dask as dxgb
      7 from xgboost.dask import DaskDMatrix

ModuleNotFoundError: No module named 'xgboost'

Solution

  • If you create a new conda environment do you get the same error? E.g.

    conda create -n kaggle_xgboost_test python=3.8 pandas numpy scikit-learn xgboost jupyter
    conda activate kaggle_xgboost_test
    
    # Then open a notebook
    jupyter-notebook
    
    # And try to import xgboost with e.g.
    import xgboost
    print("imported ok")
    

    Edit

    If the fresh environment installed/imported xgboost without any issues (as your comment suggests), then the problem you're having is unique to your existing environment. Have you resolved the conflicts identified by pipdeptree --reverse --packages xgboost? Have you tried building xgboost from source with GPU support enabled? E.g.

    git clone --recursive https://github.com/dmlc/xgboost
    cd xgboost
    mkdir build
    cd build
    cmake .. -DUSE_CUDA=ON
    make -j4