Armadillo C++ fails to diagonalize bigger matrices

Question

I'm working with the Armadillo linear algebra library for C++ to diagonalize large matrices up to 65k x 65k on a cluster operating with SLURM. For matrices larger than 30k x 30k i get the following error message:

Intel MKL ERROR: Parameter 9 was incorrect on entry to DSYTRD.

Intel MKL ERROR: Parameter 8 was incorrect on entry to DSTEDC.

Intel MKL ERROR: Parameter 12 was incorrect on entry to DORMTR.

I assume this could be caused by using int32 instead of int64, even though I enabled the arma macros ARMA_USE_64bit, ARMA_USE_BLASS_LONG_LONG etc so the internal code should operate on 64-bit integers. Here is my compilation line

    g++ main.cpp IsingModel.cpp IsingModel_disorder.cpp IsingModel_sym.cpp tools.cpp\ user_interface.cpp -o Ising.o -pthread -I../LIBRARIES_CPP/armadillo-10.8.0/include/ \
-L${MKLROOT}/lib/intel64 -fopenmp -lmkl_intel_lp64 -lmkl_core -lmkl_intel_ilp64\
 -lmkl_sequential  -lpthread -lm -lmkl_gnu_thread -lstdc++fs -llapack -fcx-fortran-rules\
 -fomit-frame-pointer -lblas -std=c++20 -std=c++17 -O3

For those, which know SLURM here are the modules I am using:

module load Armadillo/9.900.1-foss-2020a
module load imkl/2021.4.0
module load OpenBLAS/0.3.18-GCC-11.2.0

Thank you for any help in advance!

Answer 1

Sorry, I do not have enough reputation to comment below so I have to make it an answer.

During my use of Armadillo, I find the current version of Armadillo and Intel MKL does not work properly and can be buggy (Ubuntu 20). Indeed there have been a lot of reports of that but it's hard for us to do anything for it is the Armadillo library that links to these libraries. Maybe this can be an issue for the developers.

I myself find OpenBlas a good alternative to Intel MKL as the support for Armadillo. It is really efficient. Moreover, if you have an Nvidia GPU, NVBlas works well with OpenBlas. (A cmake solution to use NVBlas together with OpenBlas is detailed in my answer.) If you use cmake to install the Armadillo library, change the CmakeLists.txt file a bit to disable the detection of MKL.

CMake Solution

If MKL is installed and it is persistently giving problems during linking, Support for MKL can be disabled by editing the CMakeLists.txt file, deleting CMakeCache.txt and re-running the cmake based installation. (Armadillo README)

In CMakeLists.txt, comment out the line (Line 327) containing:

INCLUDE(ARMA_FindMKL)

g++ Solution

With g++, delete the linking library for Intel MKL, use -lopenblas for OpenBlas.

g++ main.cpp IsingModel.cpp IsingModel_disorder.cpp IsingModel_sym.cpp tools.cpp \
user_interface.cpp -o Ising.o -pthread -I../LIBRARIES_CPP/armadillo-10.8.0/include/ \
-fopenmp -lpthread -lm -lstdc++fs -llapack -fcx-fortran-rules \
-fomit-frame-pointer -lblas -lopenblas -std=c++20 -O3