Search code examples
pythontensorflowscikit-learnpython-3.8apple-m1

How to run sklearn library with native tensorflow on m1 mac


I have installed TensorFlow using a virtual environment running python 3.8 as described by Apple. This should theoretically run natively and utilise the GPU. I tried installing TensorFlow using miniforge last time and it was not able to use the GPU as miniforge uses python 3.9 and Tensorflow for m1 macs currently require python 3.8.

On sklearns website, the only way to install sklearn libraries currently is by using conda install sklearn which is through miniforge.

Is there a way to install sklearn on a tensorflow environment created using

python3 -m venv TFGPU

I have already tried pip. I was able to install most other libraries other than sklearn which I use for pre-processing.


Solution

  • Hi and welcome to SO :)

    I'm a pip/virtualvenv user, so I had to fix my venv to work with my M1 mac using the conda/miniforge, without switching to conda's venv. So, I believe this should work for you as well:

    # if not yet installed
    xcode-select --install                                      
    
    git clone git://github.com/scikit-learn/scikit-learn.git                                                                        
    cd scikit-learn                                             
    
    # mac / mac m1 specific
    brew install libomp                                         
    brew install miniforge                                      
    wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh                                                                                                                                                                      
    bash Miniforge3-MacOSX-arm64.sh                             
    
    conda init bash
    conda create -n conda-sklearn-dev -c conda-forge python numpy scipy cython joblib threadpoolctl pytest compilers llvm-openmp                                                                                                                                           
    conda activate conda-sklearn-dev                                                                         
    
    pip install cython                                     
    pip install --verbose --no-build-isolation --editable .
    

    Now:

    1. Test it on the conda venv. You should get version 0.24.2 (at the time of writing)
    2. Deactivate all conda venv-s
    3. Activate your regular venv and test again. you should get version .dev0
    4. In case sklearn is missing - do a simple pip install for it - it should grab the compiled one