Search code examples
rsnowfall

Error in running sfLapply in R

My piece of code looks like:

x<- c(1,2,3,4,5)

library(snowfall)

f1<- function(a,list){
  f2<-function(b,num){ return(abs(num-b))}

  l1<-sfLapply(list, f2, num=a)
  l1<-sum(unlist(l1))
  return(l1)
}

sfInit(parallel=TRUE,cpus=4)
l2<-(sfLapply(x, f1, list=x))
sfStop()
l2

when I run the last four lines, it gives an error:

l2<-(sfLapply(x, f1, list=x))
Error in checkForRemoteErrors(val) : 
  4 nodes produced errors; first error: could not find function "sfLapply"

When I switch to sequential processing, using lapply, it runs perfectly.

> l2<-(lapply(x, f1, list=x))
> l2
[[1]]
[1] 10

[[2]]
[1] 7

[[3]]
[1] 6

[[4]]
[1] 7

[[5]]
[1] 10

Why is sfLapply throwing an error?

Solution

  • You need to load the snowfall package on the cluster nodes. So insert 

    sfLibrary(snowfall)

    after sfInit().

    EDIT: For clarification:

    Your function f1 contains the function sfLapply, which is found in the snowfall package. When you initialize the cluster using sfInit as above, the snow package is loaded on each node of the cluster, but not the snowfall package. Without the latter, there is no object (function or otherwise) called sfLapply on the nodes, and you get the error.