Search code examples
rrvesthttr

Scrape table with rvest by clicking on button

I'm working with financial data and i want to scrape data from this site web. https://www.sikafinance.com/marches/historiques?s=BRVMAG. I want to scrape the data of a table from above site that takes three arguments

•dlPeriod

•datefrom

•dateto

And finally click on the button btcChange ="OK".

After trying the code below I only get the first table. I would like to be able to get the other tables when I change the start and end date. But unfortunately even if I change the dates I still get the table for today's date. Does anyone have any idea how to recover the whole table? I noticed when inspecting their site that when you change the dates, it doesn't change in the and tags in their code. The images are at the bottom.

I think either the whole table is available and it does a filter for the dates(gap between dates must not exceed 3 months)

library(httr)
library(rvest)

first_date<-as.Date("2022-02-01")
end_date <- as.Date("2022-03-29")
query_params <- list(dlPeriod = "Journalière",
                     datefrom = first_date, 
                     dateto = end_date,
                     btnChange = "OK")

parameter_response <- GET("https://www.sikafinance.com/marches/historiques?s=BRVMAG", query_params)

parameter_response1<- httr::content(parameter_response, as = "text", encoding = "UTF-8")
parameter_response2 <- read_html(parameter_response1)%>%
  html_node('#tblhistos')%>%
  html_table()

parameter_response2

# Date       Clôture `Plus bas` `Plus haut` Ouverture `Volume Titres` `Volume FCFA` `Variation %`
# <chr>        <chr>   <chr>      <chr>       <chr>     <chr>                   <int> <chr>        
# 1 29/04/2022 312,09  312,09     312,09      312,09    -                           0 2,53%        
# 2 28/04/2022 304,38  304,38     304,38      304,38    -                           0 0,00%        
# 3 27/04/2022 304,38  304,38     304,38      304,38    -                           0 2,69%        
# 4 26/04/2022 296,42  296,42     296,42      296,42    -                           0 0,81%        
# 5 25/04/2022 294,05  294,05     294,05      294,05    -                           0 1,34%        
# 6 22/04/2022 290,17  290,17     290,17      290,17    -                           0 0,36%

Then i change the date to see if Can get a New table but unfortunatly it does work.

first_date<-as.Date("2021-02-01")
end_date <- as.Date("2021-04-29")

After doing the same processus to scrape the New table i get only the old one.

Inspection of the web

I change the date by the old date remain in the

but the source code show there function

Solution

  • I see a POST API request called for historic data with the params as below:

    'ticker'= 'BRVMAG', 'datedeb'= '2022-02-01', 'datefin'= '2022-03-29','xperiod'= '0'

    I tested with removing headers and cookies and it seems they are not needed.

    library(httr2)
library(magrittr)

r <- request("https://www.sikafinance.com/api/general/GetHistos") %>% 
  req_body_json(list('ticker'= 'BRVMAG', 'datedeb'= '2022-02-01', 'datefin'= '2022-03-29','xperiod'= '0')) %>% 
  req_perform() %>%
  resp_body_json(simplifyVector = T)