Search code examples
rsslcurlweb-scrapingrcurl

Ignore weak DH in RCurl or curl


I used the code below to download a CSV from the url in the post, but since a recent update of my systems libcurl my curl has gotten more cautious and complains about weak DH parameters, in bash I can ignore the weak DH-parameters with the --ciphers 'DEFAULT:!DH' option. I did not find an option to the GET, fetch_memory, getURL functions in the httr, curl and RCurl packages respectively, that does the same.

How can I ignore this errors in R? (Every other well supported library also works, preferably tidyverse or something from r-opensci)

raw_data <- read_delim(
  "https://info.gesundheitsministerium.at/data/Epikurve.csv",
  delim = ";",
  col_types = cols(col_date(format="%d.%m.%Y"), col_integer(), col_datetime())
  )

Note that depending on your libcurl version you will not be able to reproduce this, older Versions seem to bee more permissive regarding weak DH-parameters. I work on Debian testing.


Solution

  • Since curl commandline utility is available on my plattform, the easiest solution I found was just using this to download the data, with the parameters that work in bash.

    raw_data <- read_delim(
      system2(
        "curl",
        args=c(
          "https://info.gesundheitsministerium.at/data/Epikurve.csv",
          "--ciphers",
          "'DEFAULT:!DH'"
        ),
        stdout=TRUE
      ),
      delim = ";",
      locale=locale(
        encoding = "UTF-8",
        date_format = "%d.%m.%Y",
        tz = "CET"
        )
      )