Search code examples
jsonrcurlhttr

Suppressing 302 Error returned by httr POST

I'm using the R httr POST function to send a body of jSON to an API. The API is, properly, returning a 302: Found message, but httr is exiting the function before I'm able to grab the body of response (which is a jSON body, with some key bits of info.)

Running httr with the Verbose() argument, the following is the response:

<- HTTP/1.1 302 Found
<- Cache-Control: no-cache
<- Pragma: no-cache
<- Content-Length: 47
<- Content-Type: application/json; charset=utf-8
<- Expires: -1
Error in function (type, msg, asError = TRUE)  : 
  necessary data rewind wasn't possible

I've run the same cURL post from terminal, and can confirm what I'm sending produces a reply from the API with both the 302 and the desired body.

For reference my R code follows. (note: y is the jSON formatted body)

POST("https://thewebsite",authenticate("myusername","mypassword",type="basic"),
    add_headers("Content-Type" = "application/json"),
    body = y, verbose())

Any thoughts on how to bypass the error and capture the 302 message content?

Solution

  • I just spent some time battling with this issue myself. The problem comes down to a difference in the HTTP spec (which is basically what RCurl adheres to) and what browsers actually do.

    The sequence of events is this:

    1. You issue POST request to server
    2. Server handles request and issues you a redirect to a new url
    3. RCurl treats this like new request like a POST, and tries to replay the body. (Browser don't try and resend the data)
    4. It can't resend the data because the underlying RCurl isn't constructed in such a way this is possible (that's why curl complains: "necessary data rewind wasn't possible")

    The solution is simple - disable the following of redirects with config(followlocation = 0L):

    POST("https://thewebsite",
  authenticate("myusername","mypassword"),
  content_type_json(),
  config(followlocation = 0L),
  body = y, 
)

# PS with httr 0.4 you can simplify to
POST("https://thewebsite",
  authenticate("myusername","mypassword"),
  config(followlocation = 0L),
  body = x, encode = "json" 
)

    You'll then need to look at the contents of the location field, and do the redirect yourself.

    For more discussion of the underlying issue, see: