Search code examples
rpipelinemagrittr

Capturing object's "safe state" within pipe chain


I'm running some scientific experiments which involve quite a lot of trial and error. What I basically do is pass some object along a chain of numerical methods with a bunch of parameters on each stage like so:

x <- define_object(param1, param2) %>%
     run_method(param3, param4) %>%
     process_results(param5) %>%
     save_plot(param6)

When everything goes smooth, I'm happy. However, each stage may fail (e.g. I modified the source and left a syntax error or the method is not compatible with the parameter set I provide), so the whole chain stops and the object is lost regardless of the steps that succeeded. In such cases I'd like to keep the object in its' latest valid state (I call that a "safe state"). I need that because each stage can take quite some time to execute.

Well, naturally, the trivial answer would be to ditch the piping paradigm and go with traditional approach:

x <- define_object(param1, param2)
x <- run_method(x, param3, param4)
x <- process_results(x, param5)
x <- save_plot(x, param6)

so I can resume the chain as soon as I fix it.

But maybe there's an option to still have pipes and keep the safe state? I'm not a number one fan of pipes, but the flow I have for this task asks for it, though the inconvenience I described is a deal-breaker.


Solution

  • Assuming you want to catch whatever went wrong, store it and proceed with calculation, tryCatch would be a natural way to handle this.

    In the following chunk I'm catching an error and outputting the state. Clearly you can implement the same logic to store the parameters somewhere as "current valid state".

    invisible(mapply(x = 1:10, y = rnorm(10), FUN = function(x, y) {
      out <- tryCatch(
        if (x == 3) {
        simpleError("o ow, something went wrong")
      } else {
        x * y
      })
    
      if (any(class(out) %in% "simpleError")) {
        message(sprintf("Something went wrong for x: %s, y: %s", x, y))
      } else {
      out
      }
    }))