How can I change the MKL (Math Kernel Library) version used by NumPy and Miniconda?
Intel's MKL doesn't perform well because on AMD processors, as MKL deliberately chooses the slowest path on non-intel CPUs, "crippling" numerical processes on AMD processors. This is very problematic for scientific work on AMD CPUs where numerical calculations using NumPy are greatly affected by this "cripple AMD" functionality.
The Python I am using (Python 3.9.5) is distributed by Miniconda, and NumPy was installed using conda install numpy. This installed NumPy with MKL version 2021.0.3.
Previously a workaround for the "cripple AMD" function was to set an environment variable MKL_DEBUG_CPY_TYPE=5. But starting MKL 2020, this variable was removed (intel, why??), so it was no longer able to get this workaround working.
So the solution now is to downgrade NumPy's (and conda's) MKL version to 2019. How can this be achieved? How can I change the MKL version used by NumPy and Conda, from 2021.0.3 to 2019?
System info:
conda: 4.10.3mkl.get_version_string()): 2021.3np.__mkl_version__): 2021.0.3Please let me know because this is a critical issue for scientific computation on AMD CPUs.
Thank you in advance!!!
PS: Before you say "MKL is written by Intel for Intel processors, so its ok to cripple other processors!", please keep in mind that there should be a competitive spirit during competition, such as amazing innovations, not anti-competitive actions like deliberately slowing down performance on competitor's CPUs . If you want to win a race, practice and improve your running technique, don't break your competitor's legs.
Please avoid a debate, and try to answer my question if you can. If you can't, just ignore and leave.
I would make a new environment, and probably source from the Anaconda channel. The following works for me:
Bash
## create environment
conda create -n foo -c anaconda python numpy mkl=2019.* blas=*=*mkl
## activate and launch python
conda activate foo
python
Python
import mkl
import numpy as np
mkl.get_version()
## 'Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications'
np.__mkl_version__
## '2019.4'
np.show_config()
## blas_mkl_info:
## libraries = ['mkl_rt', 'pthread']
## library_dirs = ['/Users/mfansler/miniconda3/envs/foo/lib']
## define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
## include_dirs = ['/Users/mfansler/miniconda3/envs/foo/include']
## blas_opt_info:
## libraries = ['mkl_rt', 'pthread']
## library_dirs = ['/Users/mfansler/miniconda3/envs/foo/lib']
## define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
## include_dirs = ['/Users/mfansler/miniconda3/envs/foo/include']
## lapack_mkl_info:
## libraries = ['mkl_rt', 'pthread']
## library_dirs = ['/Users/mfansler/miniconda3/envs/foo/lib']
## define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
## include_dirs = ['/Users/mfansler/miniconda3/envs/foo/include']
## lapack_opt_info:
## libraries = ['mkl_rt', 'pthread']
## library_dirs = ['/Users/mfansler/miniconda3/envs/foo/lib']
## define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
## include_dirs = ['/Users/mfansler/miniconda3/envs/foo/include']
I'm leaving this here as an "as-is" answer, since there seems to be some complications that are beyond me. Namely, Anaconda and Conda Forge appear to have different integration strategies when it comes to NumPy + MKL. Anaconda builds NumPy with direct integration (including the np.__mkl_version__ extension); Conda Forge appears to generically build NumPy with BLAS/LAPACK, and in turn builds libblas, liblapack variants based on MKL implementations. Not sure what differences these strategies might make.
Anaconda channel only has Python 3.8 at this point - but that fits with the intended MKL 2019.* anyway. Python 3.9 was a late 2020 release.
The blas=*=*mkl is crucial: that is what constrains to using an MKL build for NumPy.
This was on osx-64 platform - hopefully, the differences are not substantial.