Search code examples
bashxargszcat

BASH how to filenames from a file and zcat them

Suppose I have a directory with contents like this:

$ ls
file1.csv.gz
file2.csv.gz
file3.csv.gz
file4.csv.gz
listOfFiles.txt

listOfFiles.txt contains a list of the files which I want to zcat and pipe to awk to process their contents. For instance, listOfFiles.txt might contain the following:

$ cat listOfFiles.txt
file2.csv.gz
file3.csv.gz

In this case, I would want zcat to operate on file2.csv.gz and file3.csv.gz, and not on any of the other files in the directory. One way to do this is (EDIT: removed {} from xargs zcat {} based on comments):

cat listOfFiles.txt | xargs zcat | awk ...

But I am wondering if there is a more succinct way that zcat can read the names of the files from listOfFiles.txt. It would have to be able to handle an arbitrarily long list of filenames, which xargs handles fine, and ideally it would invoke zcat only once for the entire contents of listOfFiles.txt, and not once per file contained in listOfFiles.txt.

Solution

  • You can avoid cat:

    xargs zcat < listOfFiles.txt | awk ...