Search code examples
rrcurl

Difference between response of GET function and getURI

So I wanna make multiple requests to an API in order to speed up the results since in serial it is simply too slow. Therefore I am trying to use GetURI among others, however, the API does not return any value, while GET requests do!

So I tried various solutions as GETURI, GETURL and getURIAsynchronous, however, none of them returns a value. I suppose it is because the API takes a long time to process the query. I will include my test key, however, it is assumed this won't be abused

links<- c("zalando.nl", "bol.com")
key <- "SILB-DBCA-4523"
APIcall <- paste0("http://www.siteprice.org/WorthApi.aspx?type=1&key=", key, "&url=", links)

#With GET, so serial
res <-GET(APIcall[1])
res1 <- rawToChar(res$content)
as.integer(unlist(xmlToList(xmlParse(res1)))[2])

#With GetURI
res <- getURIAsynchronous(APIcall)
res1 <- rawToChar(res$content)
as.integer(unlist(xmlToList(xmlParse(res1)))

getURIAsynchronous should return the values of GET requests and it doesn't

Solution

  • Took a look at the SitePrice API. Seems like the API is using https instead of using http. And you will have your initial result for getURL as Object Moved...

    GET function will just go to your redirect page which is https page, but the default for getURI in RCurl package is to get the raw page.

    And you have two solutions here:

    1. use followlocation=TRUE option to get http redirect to https

      res <- getURIAsynchronous(APIcall, .opts=curlOptions(followlocation=TRUE))

    2. use https with getURIAsynchronous function

      APIcall <- paste0("https://www.siteprice.org/WorthApi.aspx?type=1&key=", key, "&url=", links)