Search code examples
rsavelapplyenvironment

Getting interim results from lapply()


I have some lapply code which takes a really long while to compute. Something like this:

n <- 10^7
res <- lapply(1:n, function(i){
  print(round(i/n*100, 0))
  rnorm(100)
  })

Can we somehow save the interim results while round(i/n*100, 0) is less then 100%, i.e. the code is still running?

I am running the my actual lapply() code right now and the part print(round(i/n*100, 0)) tells me that my computer is almost done (99%!). The problem is that I need to turn off the computer because I leave my work place. Is there any chance to get the data R has calculated so far while the res <- lapply(...) part is still running? Really don`t want him to calculate for ages those 99% tomorrow again..


Solution

  • Once the lapply() code runs there seems to be no way to catch the interim results. But there are two options

    • using a for loop as suggested by @GregorThomas
    • using <<- inside the lapply() function which does save interim results.

    Here is the second option for the piece of code in the question:

    n <- 10^7
    res <- vector("list", n)
    res <- lapply(1:n, function(i){
      print(round(i/n*100, 0))
      res[[i]] <<- rnorm(100)
    })
    

    Now we can have a look and see that the interim data is actually saved:

    head(res)
    tail(res)