Search code examples
rurlproxygethttr

How to use proxies with urls in R?


I am trying to use proxies with my request urls in R. It changes my requested url from "www.abc.com/games" to "www.abc.com/unsupportedbrowser"

The proxies are working since I tested them in python. However I would like to implement it in R

I tried using "httr" and "crul" library in R

#Using httr library
r <- GET(url,use_proxy("proxy_url", port = 12345, username = "abc", password ="xyz") )
text <-content(r, "text")


#using "crul"

res <- HttpClient$new(
   url,
  proxies = proxy(proxy_url:12345,"abc","xyz")
  )

out <-res$get()
text <-out$parse("UTF-8")

Is there some other way to implement the above using proxies or how can I avoid getting the request url changing from "www.abc.com/games" to "www.abc.com/unsupportedbrowser"

I also tried using "requestsR" package

However when I try something like this:

library(dplyr)
library(reticulate)
library(jsonlite)
library(requestsR)
library(rvest)
library(listviewer)
proxies <- 
  "{'http': 'http://abc:xyz@proxy_url:12345',
  'https': 'https://abc:xyz@proxy_url:12345'}" %>% 
  convert_dictionary_to_list()
res <- Get(url, proxy=proxies)

It gives an error: "r$get : $ operator is invalid for atomic vectors" I don't understand why it raises such an error. Please Let me know if this can be resolved

Thanks!


Solution

  • I was able to solve the above problem using "user_agent" argument with my GET()