Search code examples
rnohuprscriptglmnet

Error when running glmnet through Rscript and nohup

When creating a predictive model with the glmnet package in R, I observed the same error mentioned both here and here. But the proposed solution of reinstalling the glmnet package does not fix my problem.

Minimal working example my_glmnet_script.R:

#!/usr/bin/env Rscript --vanilla

# load libraries
library(methods)
library(glmnet)
library(doParallel)

# create toy model
n = 100 
p = 250 
x = matrix(rnorm(n*p), n, p)
y = matrix(rnorm(n), n, 1)

# number of parallel cores to use
ncores = 4 
registerDoParallel(ncores)

# print this before glmnet throws error
print(sessionInfo())

# fit model
my.glmnet = cv.glmnet(x=x, y=y, grouped=FALSE, parallel=TRUE, nfolds=n)
cat("end script.\n")

My command:

nohup Rscript my_glmnet_script.R > nohup.out 2> nohup.err &

Output file nohup.out:

R version 3.4.3 (2017-11-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

locale:
[1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
[3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
[5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
[7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
[9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  methods   stats     graphics  grDevices utils datasets 
[8] base     

other attached packages:
[1] doParallel_1.0.11 iterators_1.0.8   glmnet_2.0-16 foreach_1.4.3    
[5] Matrix_1.2-12    

loaded via a namespace (and not attached):
[1] compiler_3.4.3   codetools_0.2-15 grid_3.4.3       lattice_0.20-35

Error file nohup.err:

nohup: ignoring input
Loading required package: Matrix
Loading required package: foreach
Loaded glmnet 2.0-16

Loading required package: iterators
Loading required package: parallel
Error in .Fortran("get_int_parms", fdev = double(1), eps = double(1), : "get_int_parms" not available for .Fortran() for package "glmnet"
Calls: cv.glmnet -> glmnet -> glmnet.control -> .Fortran
Execution halted

Output from Rscript --version:

R scripting front-end version 3.3.3 (2017-03-06)

The confusing part is that the following command works just fine:

Rscript my_glmnet_script.R

It also runs fine if I call source("my_glmnet_script.R") from within R.

Why does nohup behave badly with glmnet and Rscript? Is there an R package that I could load that would make this work?

Solution

  • Turns out that nohup is probably not the problem.

    From my question, the R version from nohup is R v3.4.3, while Rscript --version returns R v3.3.3. This is because my $PATH points to my personal R (v3.3.3) before using the system-wide R (v3.4.3).

    Why does the system-wide R get called? The answer lies in the script hash-bang:

    #!/usr/bin/env Rscript --vanilla

    To wit, calling /usr/bin/env Rscript --version yields

    R scripting front-end version 3.4.3 (2017-11-30)

    and the system-wide R looks for glmnet in my personal (v3.3.3) library, the first entry of .libPaths(), which was never compiled for 3.4.3:

    /usr/bin/Rscript -e ".libPaths(.libPaths()[2:3]); library('glmnet')"

    Error in library("glmnet") : there is no package called ‘glmnet’

    Moral of story: stay organized when multiple R versions are installed on your server!