Search code examples
rapihttr

How can I check for 503 errors before using httr::content()?


I have run into what I think is a server error when I'm using httr. I am using the OpenLibrary Books API to extract subject data for several books by their ISBN. This code was working perfectly fine until today. Now, inconsistently, I get an exciting new error:

Error: object of type 'externalptr' is not subsettable

Every once in a while, everything runs without a hitch. While I would like to actually solve this issue and have it work every time, I think if it's a server issue I will have to make a function that is robust to this type of situation. Is there a nice way to check whether this error will occur, and return a custom error message if there is a server issue? Thank you in advance for your help.

Here is some sample code that reproduces my error:

library(httr)
library(curl)
library(purrr)

# ISBNs. The second one is the only one with data available
x <- c("0735224633", "0679745580", "1476746583", "") 

subj_list <- list()

for(i in 1:length(x)){
  isbn <- x[i]
  if(isbn =='') {
    subj_list[[i]] = NA} else{ # if missing isbn, return NA
  r <- httr::GET(paste('https://openlibrary.org/api/books?bibkeys=ISBN:',
                           isbn,
                           '&jscmd=data&format=json',sep=''))
  rl <- httr::content(r, "parsed")
  if(length(rl)==0){subj_list[[i]] = NA}else{ # if no info avail, return NA
    ilist <- rl[[1]]$subjects #get list of subjects for book i
    subj_list[[i]] <- unlist(purrr::map(ilist,'name')) 
  }
  }
}

The second book in the list is the only one with subject data available from OpenLibrary. So when the code does work, it returns the following list:

> subj_list
[[1]]
[1] NA

[[2]]
 [1] "Cas, Études de"                     "Murder"                             "Meurtre"                           
 [4] "Análisis de casos"                  "Asesinato"                          "Case studies"                      
 [7] "Murder -- Kansas -- Case studies."  "Crime"                              "Smith, Perry Edward, 1928-1965"    
[10] "Kansas"                             "Hickock, Richard Eugene, 1931-1965"

[[3]]
NULL

[[4]]
[1] NA

Here is my session info:

R version 3.6.3 (2020-02-29) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Catalina 10.15.4

Matrix products: default BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages: [1] purrr_0.3.3 curl_4.3    httr_1.4.1 

loaded via a namespace (and not attached): [1] compiler_3.6.3 magrittr_1.5   R6_2.4.1       tools_3.6.3    Rcpp_1.0.4     xml2_1.2.5 jsonlite_1.6.1 [8] packrat_0.5.0  rlang_0.4.5

Solution

  • Check the status code before trying to access the content (with a server error like 503 there likely isn't content there)

    if(httr::status_code(r) != 200) stop("Error has occurred")
    

    You can make the status code checking as elaborate as your want, with different custom error messages (maybe a different one for server 500 errors and one for user errors 400)

    List of HTTP Codes