Search code examples
pythonnumpybuildapple-m1accelerate-framework

How to build NumPy from source linked to Apple Accelerate framework?


It is my understanding that NumPy dropped support for using the Accelerate BLAS and LAPACK at version 1.20.0. According to the release notes for NumPy 1.21.1, these bugs have been resolved and building NumPy from source using the Accelerate framework on MacOS >= 11.3 is now possible again: https://numpy.org/doc/stable/release/1.21.0-notes.html, but I cannot find any documentation on how to do so. This seems like it would be an interesting thing to try and do because the Accelerate framework is supposed to be highly-optimized for M-series processors. I imagine the process is something like this:

  1. Download numpy source code folder and navigate to this folder.
  2. Make a site.cfg file that looks something like:
[DEFAULT]
library_dirs = /some/directory/
include_dirs = /some/other/directory/

[accelerate]
libraries = Accelerate, vecLib
  1. Run python setup.py build

The problem is I do not know 1. what the variables library_dirs and include_dirs should be so that NumPy knows to use Accelerate BLAS and LAPACK and 2. if there are any other additional steps that need to be taken. If anyone knows how to do this or can provide any insight, it would be greatly appreciated.


Solution

  • I actually attempted this earlier today and these are the steps I used:

    • In the site.cfg file, put
    [accelerate]
    libraries = Accelerate, vecLib
    
    • Build with NPY_LAPACK_ORDER=accelerate python3 setup.py build

    • Install with python3 setup.py install

    Afterwards, np.show_config() returned the following

    blas_mkl_info:
      NOT AVAILABLE
    blis_info:
      NOT AVAILABLE
    openblas_info:
      NOT AVAILABLE
    accelerate_info:
        extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
        extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
        define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    blas_opt_info:
        extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
        extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
        define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    lapack_mkl_info:
      NOT AVAILABLE
    openblas_lapack_info:
      NOT AVAILABLE
    openblas_clapack_info:
      NOT AVAILABLE
    flame_info:
      NOT AVAILABLE
    lapack_opt_info:
        extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
        extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
        define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    Supported SIMD extensions in this NumPy install:
        baseline = NEON,NEON_FP16,NEON_VFPV4,ASIMD
        found = ASIMDHP,ASIMDDP
        not found = 
    

    and my quick test suggest significant performance boost relative to OpenBlas.