Search code examples
rfor-loopbatch-processinghttr

Errors in Batch http status code test using for loop in r


I have a df of unique ids x urls.

library (httr)

for (i in (1:nrow(df))) {
  resp <- httr::GET(df$url[i])
  httpcode[i] <- status_code(resp)
  httpstatus[i] <- http_status(resp)$reason
}

I want to (a) find the status_code for every url, (b) find the http_status for every url, and (c) spit them out into new columns in the same df.

Problems: 1. In the code below, when I replace i by an actual index number (e.g. i = 1), the code works. When I put it in a for loop, it gives me the following error:

Error in curl::curl_fetch_memory(url, handle = handle) : 
  Couldn't resolve host name
  1. How do I make httpcode and httpstatus convert from objects into new columns in the same df? Thanks

Solution

  • out_df <- data.frame()
    for (i in df$url) {
      print(i)
      resp <- httr::GET(i)
      httpcode <- status_code(resp)
      httpstatus <- http_status(resp)$reason
      row <- c(i, httpcode, httpstatus)
      out_df <- rbind(out_df, row)
    }
    
    df <- merge(df, out_df, by = 'url', all.x = TRUE)