Search code examples
rshinyshiny-servershinyapps

How to get a dynamic url to be used to scrape data in R?


I'm working on a shinyapp that scrapes stock data from a website according to a range of dates chosen by the user. I use the package rvest to scrape my data however I'm still stuck on programming R to capture the user's date ranges and store it into a usable link.

Here is my UI

dateRangeInput("daterange", "Date range:",start  = "2000-01-01",
                                          end    = lubridate::today(),
                                          min    = "2000-01-01",
                                          max    = lubridate::today(),
                                          format = "mm/dd/yyyy",
                                          separator = "/"
               )

Here is my Server :

lien = read_html(link = quote(
                                 paste0("https://www.marketwatch.com/investing/stock/",
                                        input$choice_company,
                                        "/download-data?startDate=",
                                        input$daterange[1],
                                        "&endDate=",
                                        input$daterange[2])
                             )
                )

when i run my app it only shows for a half second and quits and the console shows this error, He can't seem to find the input, Do you have any Idea on how to solve this or any useful documentation ?

Warning: Error in ..stacktraceon..: objet 'input' introuvable
[No stack trace available]
Error in ..stacktraceon..({ : objet 'input' introuvable

The message in french : objet 'input' introuvable just means object 'input' not found

Thanks in advance


Solution

  • You do not need to use quote() on the URL. If you do that you get an unevaluated expression (as language object) instead of a string. The next step is read_html() that won't accept an argument link. Instead it's first argument is x which can be an URL. So, removing link = quote( should fix it already.

    This sample app shows the raw HTML as output to demonstrate it.

    library(shiny)
    library(rvest)
    
    ui <- fluidPage(
      textInput("choice_company", "Enter name of a company"),
      dateRangeInput("daterange", "Date range:", start  = "2000-01-01",
                     end    = Sys.Date(),
                     min    = "2000-01-01",
                     max    = Sys.Date(),
                     format = "mm/dd/yyyy",
                     separator = "/"),
      textOutput("ShowUrl"),
      hr(),
      textOutput("ShowHtml")
    )
    
    server <- function(input, output, session) {
    
      URL <- reactive({
          paste0("https://www.marketwatch.com/investing/stock/",
                 input$choice_company,
                 "/download-data?",
                 "startDate=", input$daterange[1],
                 "&endDate=",  input$daterange[2])
      })
    
      output$ShowUrl <- renderText({
        validate(need(input$choice_company, "No company selected"))
        #print(typeof(quote(paste0(input$daterange[1]))))
        URL()
      })
    
      output$ShowHtml <- renderText({
        validate(need(input$choice_company, "No company selected"))
        lien <- read_html(URL())
        paste(lien) # to convert the list into one long string
      })
    }
    
    shinyApp(ui, server)