Search code examples
rsleeppurrr

Run a function x amount of times then sleep for a period before going again


I have a data frame of 1000 values which I need to pass to an API.

I have found that the API kicks an error if I pass more than 500 values so I wish to pass say 400 values, sleep for 10 mins before passing another 400 values and then going to sleep for another 10 mins before finishing with the final 200.

In order to provide a reprex here is a small sample of data:

examples <- data.frame(names = c(1003060377,1003213240,1003116930,1003020306,1003292350,1003094988,1003164716,1003156324,1003219981))

install.packages("pacman")

pacman::p_load(tidyverse,devtools)

devtools::install_github("frankfarach/npi")

x <- map_dfr(examples$names,npi::npi_search) %>% 
  select(addresses) %>% 
  unnest()

Given the small sample size above if I could get the function to sleep after passing two values for 1 min before going onto the next two etc. until all values have been passed.

If anyone can help I would be very appreciative.


Solution

  • You can use a for loop so that you can keep track of entries that has been processed.

    library(dplyr)
    library(tidyr)
    
    result <- vector('list', nrow(examples))
    
    for(i in seq(nrow(examples))) {
      #Sleep for 1 minute after every 2 values
      if(i %% 2 == 0) Sys.sleep(60)
      result[[i]] <- npi::npi_search(examples$names[i])
    }
    
    bind_rows(result) %>%
      select(addresses) %>% 
      unnest(addresses)
    
    #  country_code country_name address_purpose address_type address_1
    #   <chr>        <chr>        <chr>           <chr>        <chr>    
    # 1 US           United Stat… LOCATION        DOM          6601 PHO…
    # 2 US           United Stat… MAILING         DOM          800 MARS…
    # 3 US           United Stat… LOCATION        DOM          5835 NE …
    # 4 US           United Stat… MAILING         DOM          2100 DOU…
    # 5 US           United Stat… LOCATION        DOM          1163 BLA…
    # 6 US           United Stat… MAILING         DOM          1163 BLA…
    # 7 US           United Stat… LOCATION        DOM          500 COMM…
    # 8 US           United Stat… MAILING         DOM          150 SCHA…
    # 9 US           United Stat… LOCATION        DOM          6420 CLA…
    #10 US           United Stat… MAILING         DOM          6420 CLA…
    #11 US           United Stat… LOCATION        DOM          118 W NO…
    #12 US           United Stat… MAILING         DOM          PO BOX M 
    #13 US           United Stat… LOCATION        DOM          520 S SA…
    #14 US           United Stat… MAILING         DOM          520 S SA…
    #15 US           United Stat… LOCATION        DOM          910 S. H…
    #16 US           United Stat… MAILING         DOM          24 ROY S…
    #17 US           United Stat… LOCATION        DOM          1595 HAR…
    #18 US           United Stat… MAILING         DOM          1595 HAR…
    # … with 6 more variables: address_2 <chr>, city <chr>, state <chr>,
    #   postal_code <chr>, telephone_number <chr>, fax_number <chr>