Search code examples
rforeachcatsink

How to avoid 'sink stack is full' error when sink() is used to capture messages in foreach loop


In order to see the console messages output by a function running in a foreach() loop I followed the advice of this guy and added a sink() call like so:

   library(foreach)    
   library(doMC)
   cores <- detectCores()
   registerDoMC(cores)

   X <- foreach(i=1:100) %dopar%{
   sink("./out/log.branchpies.txt", append=TRUE)
   cat(paste("\n","Starting iteration",i,"\n"), append=TRUE)
   myFunction(data, argument1="foo", argument2="bar")
   }

However, at iteration 77 I got the error 'sink stack is full'. There are well-answered questions about avoiding this error when using for-loops, but not foreach. What's the best way to write the otherwise-hidden foreach output to a file?


Solution

  • This runs without errors on my Mac:

    library(foreach)    
    library(doMC)
    cores <- detectCores()
    registerDoMC(cores)
    
    X <- foreach(i=1:100) %dopar%{
      sink("log.branchpies.txt", append=TRUE)
      cat(paste("\n","Starting iteration",i,"\n"))
      sink() #end diversion of output
      rnorm(i*1e4)
    }
    

    This is better:

    library(foreach)    
    library(doMC)
    cores <- detectCores()
    registerDoMC(cores)
    sink("log.branchpies.txt", append=TRUE)
    X <- foreach(i=1:100) %dopar%{
      cat(paste("\n","Starting iteration",i,"\n"))
        rnorm(i*1e4)
    }
    sink() #end diversion of output
    

    This works too:

    library(foreach)    
    library(doMC)
    cores <- detectCores()
    registerDoMC(cores)
    
    X <- foreach(i=1:100) %dopar%{
      cat(paste("\n","Starting iteration",i,"\n"), 
           file="log.branchpies.txt", append=TRUE)
      rnorm(i*1e4)
    }