Can't add error cases to list R using tryCatch()

Question

I am trying to call an API that throttles data use for requests that are too large. I have broken up the data to respect all the terms of use. What I want to do is call the API on what I think are reasonable chunks, however if they throw an error, get the offending chunk back and then break this up further. I can generate a list of viable data, but not the list of failed requests. Here is a mock up of what I am trying:

#create the data
a <- 1
b <- 2
c <- 'three'
d <- 4

ls <- list(a, b, c, d)

#create a function that will throw one error
add <- function(x){
  output <- x + 1
  return(output)
}

#Call a for loop that will populate two lists - one of viable calls and one of error cases

new_ls <- list()
errors_ls <- list()
for (i in 1:length(ls)) {
  tryCatch({new_ls <- append(new_ls, list(add(ls[[i]])))}
           , error = function(e) {errors_ls <- append(errors_ls, list(ls[[i]]))}) 
}
print(new_ls)
print(errors_ls)

Gives:

> print(new_ls)
[[1]]
[1] 2

[[2]]
[1] 3

[[3]]
[1] 5

> print(errors_ls)
list()

Notably errors_ls is empty. What I was expecting is:

[[1]]
[1] "three"

I appreciate that I should be doing this with apply. The API call is however really messy (I also artificially limit the frequency of calls, so speed isn't an issue), so I find it easier to iterate over the API calls in a for loop. I have tried following the documentation on tryCatch, including playing with the structure of the tryCatch({}) syntax, based on other posts on this, but I can't get it right.

Answer 1

There are couple of ways to get the output. In the OP's code, the errors_ls is getting assigned within the function env, and it is not updating the object 'errors_ls' in the global env. We can use <<- instead of <- to make the change

 new_ls <- list()
 errors_ls <- list()
   for (i in 1:length(ls)) {
     tryCatch({new_ls <- append(new_ls, list(add(ls[[i]])))}
              , error = function(e) {
        errors_ls <<- append(errors_ls, list(ls[[i]]))}) 
   }

-checking

> new_ls
[[1]]
[1] 2

[[2]]
[1] 3

[[3]]
[1] 5

> errors_ls
[[1]]
[1] "three"

---

Or another option is to make changes in the loop to do the assignment outside

 new_ls <- list()
  errors_ls <- list()
  for (i in 1:length(ls)) {
   tmp <- tryCatch({list(add(ls[[i]]))}
             , error = function(e) {return(list(ls[[i]]))})
    if(is.numeric(unlist(tmp)))
    new_ls <- append(new_ls, tmp)
    else errors_ls <- append(errors_ls, tmp)
  }

-checking

> new_ls
[[1]]
[1] 2

[[2]]
[1] 3

[[3]]
[1] 5

> errors_ls
[[1]]
[1] "three"