I found that using one of BLAS/ATLAS/MKL/OPENBLAS will give improvement on speed in R. However, will it still improve the R Package that is written in C or C++? for example, R package Glmnet is implemented in FORTRAN and R package rpart is implemented in C++. Will it just installing BLAS/...etc will improve the execution time? or do we have to rebuild (building new C code) the package based on BLAS/...etc?
It is frequently stated, including in a comment here, that "you have to recompile R" to use different BLAS or LAPACK library. That is wrong.
You do not have to recompile R provided it is build against the shared library versions of BLAS and LAPACK.
I have a package and vignette on CRAN which uses this fact to provide a benchmarking framework in which different BLAS and LAPACK version are timed against each just by installing different ones (one commmand in Debian/Ubuntu) and running benchmarks -- this is so straightforward that it can be automated in a package such as this.
The results in that package will provide an idea of the possible speed differences. Exactly how they pan out depends on your computer, your data (size), your problem etc. But if, say, your problem uses LAPACK functions which can run benefit from running multithreaded then installing OpenBLAS may help. That is true for any R package using LAPACK as they will use the same LAPACK installation accessed through are, and these can be changed.