Search code examples
pythonscipyanacondalapack

Error when diagonalising large matrices using anaconda scipy

I have recently switched from using homebrew python on mac OS X to using anaconda, and I have started getting an error when diagonalising large(ish) matrices. Calling scipy.linalg.eigvalsh(A) with matrices above about 3000x3000 entries gives an error:

$HOME/anaconda2/lib/python2.7/site-packages/scipy/linalg/decomp.pyc in eigvalsh(a, b, lower, overwrite_a, overwrite_b, turbo, eigvals, type, check_finite)
    762                 overwrite_a=overwrite_a, overwrite_b=overwrite_b,
    763                 turbo=turbo, eigvals=eigvals, type=type,
--> 764                 check_finite=check_finite)
    765 
    766 

$HOME/anaconda2/lib/python2.7/site-packages/scipy/linalg/decomp.pyc in eigh(a, b, lower, eigvals_only, overwrite_a, overwrite_b, turbo, eigvals, type, check_finite)
    385         if eigvals is None:
    386             w, v, info = evr(a1, uplo=uplo, jobz=_job, range="A", il=1,
--> 387                              iu=a1.shape[0], overwrite_a=overwrite_a)
    388         else:
    389             (lo, hi) = eigvals

ValueError: On entry to ZHBRDB parameter number 12 had an illegal value

The final error message seems similar to this old scipy issue: scipy/issues/5401, but the matrices I'm having problems with are much smaller.

Running print np.__config__.show() gives:

lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$HOME/anaconda2/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['$HOME/anaconda2/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$HOME/anaconda2/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['$HOME/anaconda2/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$HOME/anaconda2/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['$HOME/anaconda2/include']
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['$HOME/anaconda2/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['$HOME/anaconda2/include']
None
Solution

  • This is, as it stands, an MKL bug which has been kindly reported by @brd490 as per the discussion in SciPy issue 8205 and is accepted by the Intel MKL team

    Note that OpenBLAS-built SciPy does not have this issue. However, in the mean time, we have also noticed that the optimal workspace array sizes were not correct in the scipy.linalg.eigh.That is also going to be corrected once the decision over how to change the signature of the the underlying ?SYEVR/?HEEVR wrappers.

    As a bonus, probably one would be able to compute the eigenvalues selectively since originally these routines allow for it but were not exposed in scipy.linalg.eigh.