Search code examples
ruser-agenthttr

Failing to set the user agent via httr::user_agent


Is there anything special I need to consider when trying to change the user agent via httr::user_agent in a httr::GET() call on MS Windows? I'm using R-3.1.0 and httr 0.3.

Following the example at ?user_agent, I'm getting these results:

url_this <- "http://httpbin.org/user-agent"

Standard user agent:

GET(url_this)   
Response [http://httpbin.org/user-agent]
  Status: 200
  Content-type: application/json
{
  "user-agent": "curl/7.19.6 Rcurl/1.95.4.1 httr/0.3"
} 

Modified user agent:

GET(url_this, user_agent("Mozilla/5.0"))
Response [http://httpbin.org/user-agent]
  Status: 200
  Content-type: application/json
{
  "user-agent": "curl/7.19.6 Rcurl/1.95.4.1 httr/0.3"
}

I had expected that the second call returns something closer to what I'm getting when visiting url_this in my browser:

{
  "user-agent": "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0"
}

What am I missing here? Also ran setInternet2(TRUE) first, but got identical results.


Solution

  • Very curious the help page ?user_agent suggests it should work. You can set a header explicitly and it does work

    > GET("http://httpbin.org/user-agent", add_headers("user-agent" = "Mozilla/5.0"))
    Response [http://httpbin.org/user-agent]
      Status: 200
      Content-type: application/json
    {
      "user-agent": "Mozilla/5.0"
    } 
    

    but the example given in ?user_agent appears not to.

    > GET("http://httpbin.org/user-agent", user_agent("Mozilla/5.0") )
    Response [http://httpbin.org/user-agent]
      Status: 200
      Content-type: application/json
    {
      "user-agent": "curl/7.19.6 Rcurl/1.95.4.1 httr/0.3"
    } 
    > 
    

    It is returning

    > httr:::default_ua()
    [1] "curl/7.19.7 Rcurl/1.95.4.1 httr/0.3"
    

    My ISP was also doing something funky so you may need:

    GET("http://httpbin.org/user-agent", add_headers("user-agent" = "Mozilla/5.0", "Cache-Control" = "no-cache"))