Search code examples
rpurrrhttr2

Using httr2::last_response() in conjunction with purrr::possibly()

I have an issue when working with httr2::last_response() and purrr::possibly() and I can't quite figure out how to resolve it, as other posts involving either httr2 or purrr::possibly doesn't really get into this combination.

An example below that works perfectly:

url <- stringr::str_c("https://mutalyzer.nl/api/normalize/", "GRCh37", "(", "NM_000546.5", "):", "c.673A>T")

url |> httr2::request() |> httr2::req_perform()
#> Error in `httr2::req_perform()`:
#> ! HTTP 422 Unprocessable Entity.
#> This is totally expected!

httr2::last_response()
#> <httr2_response>
#> GET https://mutalyzer.nl/api/normalize/GRCh37(NM_000546.5):c.673A>T
#> Status: 422 Unprocessable Entity
#> Content-Type: application/json
#> Body: In memory (1608 bytes)

This makes perfect sense, since httr2::last_response() capture/keep the last response regardless of the status of the request.

I wanted to take advantage of this feature to eventually iterate over multiple url's and always get back something I could work with, so I thought I could implement purrr::possibly() with httr2::last_response() like so:

unloadNamespace("httr2") # Unloading httr2 namespace to 'reset' (in lack of better word) httr2::last_response()

url <- stringr::str_c("https://mutalyzer.nl/api/normalize/", "GRCh37", "(", "NM_000546.5", "):", "c.673A>T")

safe_request <- purrr::possibly(\(url) url %>% httr2::request() %>% httr2::req_perform(), otherwise = httr2::last_response(), quiet = TRUE)

result <- safe_request(url) %>% httr2::resp_body_json()
#> Error in `httr2::resp_body_json()`:
#> ! `resp` must be an HTTP response object, not `NULL`.

But as can be seen it tells us that the response (what comes from running safe_request(url)) is NULL, and so it puzzles me; why doesn't my safe_request function first run:

url %>% httr2::request() %>% httr2::req_perform()"

followed by

httr2::last_response()

when the first part results in a HTTP 422 error? And is there any way I can achieve the desired output (as shown in first code sample) in some way, that makes for a good scalable function to use with the map-family of purrr?

I'm trying to descend to deeper levels understanding on the subject of environments and complex iteration, but I'm grasping sometimes, so bare with me if this seems to be a overcomplicated hard way to do, what I'm trying to do.

Solution

  • The answer here becomes obvious if we examine the code of purrr::possibly

    purrr::possibly
#> function (.f, otherwise = NULL, quiet = TRUE) 
#> {
#>     .f <- as_mapper(.f)
#>     force(otherwise)
#>     check_bool(quiet)
#>     function(...) {
#>         tryCatch(.f(...), error = function(e) {
#>             if (!quiet) 
#>                 message("Error: ", conditionMessage(e))
#>             otherwise
#>         })
#>     }
#> }

    It's a fairly short and straightforward function factory that takes as arguments any arbitrary function .f and any arbitrary R object otherwise. It returns a new function which takes the same arguments as .f did, but will run .f inside tryCatch, such that if an error occurs, otherwise is returned instead of the program stopping.

    The line force(otherwise) means that the otherwise argument is evaluated and stored as the value it had upon the creation of the safe_request function.

    That means when you create safe_request you are "locking in" the value of httr2::last_response() at that time. Since you are creating safe_request after unloading httr2, this value is NULL.

    To demonstrate this, let us use possibly to create safe_request in a new session.

    safe_request <- purrr::possibly(\(url) url %>% 
                                  httr2::request() %>% 
                                  httr2::req_perform(), 
                                otherwise = httr2::last_response(), quiet = TRUE)

    Here, possibly has created safe_request as a closure, that is, a function with an associated evaluation environment. The evaluation environment contains variables called .f and otherwise that were locked in when safe_request was created.

    safe_request
#> function (...) 
#> {
#>     tryCatch(.f(...), error = function(e) {
#>         if (!quiet) 
#>             message("Error: ", conditionMessage(e))
#>         otherwise
#>     })
#> }
#> <bytecode: 0x000002c1d3559cd8>
#> <environment: 0x000002c1d355cd18>

    We can examine the value of .f and otherwise as follows:

    environment(safe_request)$.f
#> \(url) url %>% 
#>                                   httr2::request() %>% 
#>                                   httr2::req_perform()
environment(safe_request)$otherwise
#> NULL

    Any time we try to use safe_request we will get the output from the function .f if it runs without error, and otherwise we will get NULL. In other words, possibly cannot be used in this way.

    However, we can take inspiration from it to write a wrapper around tryCatch that will only run the otherwise code if the request fails:

    safe_request <- function(url) {
  tryCatch(url |> httr2::request() |> httr2::req_perform(),
           error = function(e) httr2::last_response())
}

url <- "https://mutalyzer.nl/api/normalize/GRCh37(NM_000546.5):c.673A>T"

result <- safe_request(url) |> httr2::resp_body_json()

    Now we get the response without a problem. Here are the first few lines:

    result 
#> $custom
#> $custom$corrected_description
#> [1] "NC_000017.10(NM_000546.5):c.673A>T"
#> 
#> $custom$corrected_model
#> $custom$corrected_model$coordinate_system
#> [1] "c"
#> 
#> $custom$corrected_model$reference
#> $custom$corrected_model$reference$id
#> [1] "NC_000017.10"
#> 
#> $custom$corrected_model$reference$selector
#> $custom$corrected_model$reference$selector$id
#> [1] "NM_000546.5"