POST to API using httr in R results in error

Question

I'm trying to pull data directly from an API into R using the httr package. The API doesn't require any authentication, and accepts JSON strings of lat, long, elevation, variable sets, and time period to estimate climate variables for any location. This is my first time using an API, but the code below is what I've cobbled together from various Stack Overflow posts.

library(jsonlite)
library(httr)    
url = "http://apibc.climatewna.com/api/clmApi"
body <- data.frame(lat = c(48.98,50.2), ##two example locations
                   lon = c(-115.02, -120),
                   el = c(1000,100),
                   prd = c("Normal_1961_1990.nrm","Normal_1961_1990.nrm"),
                   varYSM = c("Y","SST"))
requestBody <- toJSON(list("output" = body),auto_unbox = TRUE) ##convert to JSON string

result <- POST("http://apibc.climatewna.com/api/clmApi", ##post to API
               body = requestBody,
               add_headers(`Content-Type`="application/json"))
content(result)

I've tried various different versions of this (e.g. writing the JSON string manually, putting the body as a list in POST with encode = "json"), and it always runs, but the content always contains the below error message:

$Message
[1] "An error has occurred."

$ExceptionMessage
[1] "Object reference not set to an instance of an object."

$ExceptionType
[1] "System.NullReferenceException"

If I use GET and specify the variables directly in the URL

url = "http://apibc.climatewna.com/api/clmApi/LatLonEl?lat=48.98&lon=-115.02&el=1000&prd=Normal_1961_1990&varYSM=Y"
result <- GET(url)
content(result)

it produces the correct output, but then I can only obtain information for one location at a time. There isn't currently any public documentation about this API as it's very new, but I've attached a draft of the section explaining it using JS below. I would very much appreciate any help/suggestions on what I'm doing wrong! Thank you!

Answer 1

The main problem is that jQuery.ajax encodes the data using jQuery.param before sending it to the API, so what it's sending looks something like [0][lat]=48.98&[0][lon]=-115.02.... I don't know of a package in R that does a similar encoding as jQuery.param, so we'll have to hack something together.

Modifying your example slightly:

library(httr)
body <- data.frame(lat = c(48.98,50.2), ##two example locations
                   lon = c(-115.02, -120),
                   el = c(1000,100),
                   prd = c("Normal_1961_1990","Normal_1961_1990"),
                   varYSM = c("Y","Y"))

Now, we do the encoding, like so:

out <- sapply(1:nrow(body), function(i) {
  paste(c(
    paste0(sprintf("[%d][lat]", i - 1), "=", body$lat[i]),
    paste0(sprintf("[%d][lon]", i - 1), "=", body$lon[i]),
    paste0(sprintf("[%d][el]", i - 1), "=", body$el[i]),
    paste0(sprintf("[%d][prd]", i - 1), "=", body$prd[i]),
    paste0(sprintf("[%d][varYSM]", i - 1), "=", body$varYSM[i])
  ), collapse = "&")
})
out <- paste(out, collapse = "&")

so now out is in a form that the API likes. Finally

result <- POST(url = "http://apibc.climatewna.com/api/clmApi", ##post to API
               body = out,
               add_headers(`Content-Type`="application/x-www-form-urlencoded"))

noting the Content-Type. We get

df <- do.call(rbind, lapply(content(result), as.data.frame, stringsAsFactors = FALSE))
str(df)
# 'data.frame': 2 obs. of  29 variables:
#  $ lat    : chr  "48.98" "50.2"
#  $ lon    : chr  "-115.02" "-120"
#  $ elev   : chr  "1000" "100"
#  $ prd    : chr  "Normal_1961_1990" "Normal_1961_1990"
#  $ varYSM : chr  "Y" "Y"
#  $ MAT    : chr  "5.2" "8"
#  $ MWMT   : chr  "16.9" "20.2"
#  $ MCMT   : chr  "-6.7" "-5.6"
#  $ TD     : chr  "23.6" "25.7"
#  $ MAP    : chr  "617" "228"
#  $ MSP    : chr  "269" "155"
#  $ AHM    : chr  "24.7" "79.1"
#  $ SHM    : chr  "62.9" "130.3"
#  $ DD_0   : chr  "690" "519"
#  $ DD5    : chr  "1505" "2131"
#  $ DD_18  : chr  "4684" "3818"
#  $ DD18   : chr  "60" "209"
#  $ NFFD   : chr  "165" "204"
#  $ bFFP   : chr  "150" "134"
#  $ eFFP   : chr  "252" "254"
#  $ FFP    : chr  "101" "120"
#  $ PAS    : chr  "194" "34"
#  $ EMT    : chr  "-36.3" "-32.7"
#  $ EXT    : chr  "37.1" "41.2"
#  $ Eref   : chr  "14.7" "13.6"
#  $ CMD    : chr  "721" "862"
#  $ MAR    : chr  "347" "679"
#  $ RH     : chr  "57" "57"
#  $ Version: chr  "ClimateBC_API_v5.51" "ClimateBC_API_v5.51"