Search code examples
bashawkzcat

Bash script pass find files to awk [Mac]

I have some gzipped log files in a directory and they look like this:

log/day_1_time_log_1.log.gz
...
log/day_1_time_log_100.log.gz
log/day_1_location_log_1.log.gz
...
log/day_1_location_log_100.log.gz

I would like to take the 4th column (some json strings) from all logs containing the string time and cat them into one file. This is what I did and I am getting zcat: unknown compression format error.

find logs/* -name *time* | zcat | awk -F"\t" '{ print $4 }' > output.json

What is wrong with my code? Can I pass that directly into awk?

Solution

  • You can use xargs:

    find logs/ -name '*time*.log.gz' -print0 | 
    xargs -0 -I % gzcat % | awk -F'\t' '{print $4}' > output.json