I want to initialize libraries in cluster by their names represented as strings.
This code works fine:
library(snowfall, rlecuyer, rsprng)
sfInit(parallel = TRUE, cpus = 4, type = "SOCK")
sfClusterEval(library(e1071))
And this code produces en error: 4 nodes produced errors; first error: object 'expr' not found
library(snowfall, rlecuyer, rsprng)
sfInit(parallel = TRUE, cpus = 4, type = "SOCK")
lib <- "e1071"
expr <- parse(text=paste("library(", lib, ")", sep=""))
sfClusterEval(expr)
So sfClusterEval
try to evaluate expr
and not an expression which expr
contains. I cannot undestand which type of expression should be passed to sfClusterEval
function, which uses substitute
in its body
> sfClusterEval
function (expr, stopOnError = TRUE)
{
sfCheck()
if (sfParallel()) {
return(sfClusterCall(eval, substitute(expr), env = globalenv(),
stopOnError = stopOnError))
}
else {
return(eval(expr, envir = globalenv(), enclos = parent.frame()))
}
}
This question seems simple, but I could not solve it and need someone's advice.
UPDATE:
Further investigation details on simplier examples. I feel that the truth is near. This code works fine
sfClusterEval(library("e1071"))
But this call produces en error: 4 nodes produced errors; first error: object 'lib' not found
lib <- "e1071"
sfClusterEval(library(lib, character.only=TRUE))
ANSWER:
The variable lib
should be exported to the cluster previously. And after that it can be removed.
lib <- "e1071"
sfExport("lib")
sfClusterEval(library(lib, character.only=TRUE))
sfRemove("lib")
Thanks for Richie, for giving the starting idea!
You can use sfLibrary
to load extra packages on workers. See ?snowfall
and click snowfall-tools
.