Search code examples
htmlrjsontidyverservest

Dynamic Webscraping in R

I am trying to scrape information about orbits for asteroids in R. I have tried rvest and selectorgadget, however the website is dynamic. The website is: https://ssd.jpl.nasa.gov/tools/sbdb_lookup.html#/?sstr=2006%20WP1

I am wanting to get the data for the Osculating Orbital Elements under the Orbit Parameters drop down shown here: enter image description here

I am not very familiar with html or json, so I am looking for help downloading this table into R.

Solution

  • Use their API.

    As an example of how to use the API, inspecting the page we can see the parameters used and recreate them (although I have changed it to use the direct identifier rather than a search string because it is quicker):

    # Query parameters
params <- list(
  spk              = "3359266",
  `alt-des`        = 1,
  `alt-orbits`     = 1,
  `ca-data`        = 1,
  `ca-time`        = "both",
  `ca-tunc`        = "both",
  `cd-epoch`       = 1,
  `cd-tp`          = 1,
  discovery        = 1,
  `full-prec`      = 1,
  `nv-fmt`         = "both",
  `orbit-defs`     = 1,
  `phys-par`       = 1,
  `r-notes`        = 1,
  `r-observer`     = 1,
  `radar-obs`      = 1,
  sat              = 1,
  `vi-data`        = 1,
  www              = 1
)

# Make query string
param_string <- paste0(names(params), "=", params, collapse = "&")

    Now retrieve the data:

    dat <- jsonlite::fromJSON(
  sprintf("https://ssd-api.jpl.nasa.gov/sbdb.api?%s", param_string)
  )

# Inspect 
dat$orbit$elements

       sigma  name                                                 title units                value  label
1   .0010672     e                                          eccentricity  <NA>    .6067433473394845      e
2   .0036921     a                                       semi-major axis    au    1.706954591882637      a
3  .00037377     q                                   perihelion distance    au    .6712712490472621      q
4   .0093907     i inclination; angle with respect to x-y ecliptic plane   deg    5.896394792692075      i
5  .00085394    om                       longitude of the ascending node   deg    234.2975070934102   node
6   .0079426     w                                argument of perihelion   deg    98.19292045631209   peri
7    .086629    ma                                          mean anomaly   deg    22.10748390629943      M
8    .033722    tp                            time of perihelion passage   TDB 2454008.477177134853     tp
9       <NA> tp_cd                            time of perihelion passage   TDB 2006-Sep-29.97717713     tp
10    2.6428   per                               sidereal orbital period     d    814.5755667075963 period
11  .0014339     n                                           mean motion deg/d    .4419479477577152      n
12  .0059322    ad                                     aphelion distance    au    2.742637934718012      Q