Search code examples
rparallel-processingcluster-computinglocale

Is there a way to set locales in cluster in r?


I have to run a piece of code using parallel processing to reduce the execution time. Each steps are independent of each other. I have created a cluster,

 cl = makeCluster(cores)
 registerDoParallel(cl)
 invisible(clusterEvalQ(cl, sapply(list.files("./Src/LibrariesAndFunctions/",
                                             full.names = TRUE,
                                             recursive = TRUE),
                                  function(x) {
                                    source(x, encoding = "UTF-8")
                                  })))

Now I need to read a set of functions available in the same directory. The problem is that the data is in japanese and I have set the locale globally outside the cluster as

Sys.setlocale("LC_ALL","japanese_JAPAN")

This works outside the cluster but not within the cluster and it throws an error like this,

2020/02/12 20:10:27.742   [INFO ]   2020/02/12 20:10:27.497   |main        |  [xxx] 結合 開始
Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  4 nodes produced errors;irst error: ./Src/LibrariesAndFunctions//01_Setup_Logging_Milestone/I103_Setup.R:15:0: unexpected end of input
13:   ##
14:   ## 
   ^

This error tells that there is an unexpected end of input at line 14 but there are japanese comments there. This can be solved only if the comments are translated to english. So my question here is, How to set the locale in a cluster node?


Solution

  • You can run arbitrary R code on each cluster node with clusterEvalQ.

    parallel::clusterEvalQ(cl, Sys.setlocale("LC_ALL","japanese_JAPAN"))