Search code examples
rcmddata.tablelapplyunzip

R: fread many zipped files via rbindlist + cmd


I am trying to read a number of zipped files into one data table via rbindlist. In older versions of data.table(), this worked, and it still works, however now it gives the following warning on each file.

data.table::rbindlist(lapply(paste0("unzip -p ",filelist), data.table::fread,fill=T),idcol=T)

Taking input= as a system command ('unzip -p ****') and a variable has been used in the expression passed to input=. Please use fread(cmd=...). There is a security concern if you are creating an app, and the app could have a malicious user, and the app is not running in a secure environment; e.g. the app is running as root. Please read item 5 in the NEWS file for v1.11.6 for more information and for the option to suppress this message.

I understand the risks and would like to suppress the messages. I see two ways to do that: 1) turn on a flag to suppress - but I can't find an option in the manuals; 2) give the filenames to the fread as explicitly cmd= . However I'm not sure how to efficiently loop through a list here.

Sorry that the example is not reproducible, but the idea should be clear - any suggestions? Thanks!


Solution

  • The right way to call it will be to use:

    data.table::rbindlist(lapply(paste0("unzip -p ", filelist),
                                 function(x) data.table::fread(cmd = x, fill=TRUE)), 
                          idcol=TRUE)