I have a code block, to perform 3 times retry of the code execution in case of a specific error. In below example if HTTP 503 error occurred during the data download from ADLS container, I want the same operation to be executed maximum of 3 times retry.
require(AzureStor)
require(stringr)
recheck <- 0
while (recheck < 3){
recheck <- recheck + 1
tryCatch({
storage_download(container, file, filename, overwrite=TRUE)
recheck <- 4
}, error = function(e){
if ( sum(str_detect(e, '503')*1) > 0 ){
print(e)
print(paste0('An infra-level failure occured. Retry sequence number is : ', recheck))
} else{
recheck <<- 4
print(e)
}
}
)
}
This code works fine for me, but similar to storage_download
in the above example, I have other ADLS operations like delete_blob
, upload_blob
, storage_upload
, list_storage_files
at multiple instances in the code, I have to write above mentioned code for each of these functions. I want to make the above code as a function which can be called during each of these ADLS operations. Any thoughts or suggestions would help me greatly.
The following should do the trick:
with_retries_on_failure = function (expr, retries = 3L) {
expr = substitute(expr)
for (try in seq_len(retries)) {
tryCatch(
return(eval.parent(expr)),
error = \(e) {
if (str_detect(conditionMessage(e), '503')) stop(e)
message('An infra-level failure occurred. Retry sequence number is: ', try)
}
)
}
}
Used as follows:
with_retries_on_failure(storage_download(container, file, filename, overwrite=TRUE))
Note the return()
call, which immediately returns from the surrounding function without the need to update the loop variable. Likewise, in the case of a failure we also don’t have to update the loop variable since we are using a for
loop, and we use stop()
to break out of the loop for any error that is not a 503 HTTP response.