I would like to download the 10-year federal note yield from the treasury website: https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield
To parse the above webpage and retrieve the most recent 10 yr treasury note yield, I used to follow the instructions provided here: Parsing 10-year federal note yield from the website
library(httr)
URL = "https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield"
urldata <- GET(URL)
data <- readHTMLTable(rawToChar(urldata$content),
stringsAsFactors = FALSE)
data <- as.data.frame((data[69]))
names(data) <- gsub("NULL.","", names(data)) # Take out "NULL."
But it no longer works.
Any thoughts what may be wrong or alternative suggestions?
This does not answer the specific question of why your code no longer works. Here is an alternative using rvest package which simplifies many scraping operations. In particular below, the selection of the table is made with the class id CSS selector .t-chart. This makes it much more tolerant of page formatting changes. The chaining with operator %>% makes for very compact code.
library(rvest)
t_url = "https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield"
rates <- read_html(t_url) %>%
html_node(".t-chart") %>%
html_table()
rates
# Date 1 mo 3 mo 6 mo 1 yr 2 yr 3 yr 5 yr 7 yr 10 yr 20 yr 30 yr
# 1 04/03/17 0.73 0.79 0.92 1.02 1.24 1.47 1.88 2.16 2.35 2.71 2.98
# 2 04/04/17 0.77 0.79 0.92 1.03 1.25 1.47 1.88 2.16 2.36 2.72 2.99
# 3 04/05/17 0.77 0.80 0.93 1.03 1.24 1.44 1.85 2.14 2.34 2.71 2.98
# 4 04/06/17 0.78 0.79 0.94 1.05 1.24 1.45 1.87 2.15 2.34 2.72 2.99
# 5 04/07/17 0.77 0.82 0.95 1.08 1.29 1.52 1.92 2.20 2.38 2.74 3.00
# 6 04/10/17 0.77 0.82 0.97 1.07 1.29 1.52 1.91 2.18 2.37 2.72 2.99
# 7 04/11/17 0.74 0.82 0.94 1.05 1.24 1.45 1.84 2.11 2.32 2.67 2.93
# 8 04/12/17 0.77 0.81 0.95 1.04 1.24 1.44 1.81 2.09 2.28 2.65 2.92
# 9 04/13/17 0.76 0.81 0.94 1.03 1.21 1.40 1.77 2.05 2.24 2.62 2.89