Search code examples
bashunixzcat

zcat filename expansion in for loop

I got these files:

bonds.txt.gz
bonds_201307.txt.gz
bonds_201308.txt.gz
bonds_201309.txt.gz
bonds_201310.txt.gz
bonds_201311.txt.gz
bonds_201312.txt.gz

I have a for loop to go through the files, uncompress them, and find a string with grep:

for f in `ls *.txt.gz`; do echo $f; zcat $f | grep MYBOND; done

If I run this I get the following:

bonds.txt.gz
zcat: bonds.txt.gz.gz: No such file or directory
bonds_201307.txt.gz
zcat: bonds_201307.txt.gz.gz: No such file or directory
bonds_201308.txt.gz
zcat: bonds_201308.txt.gz.gz: No such file or directory
bonds_201309.txt.gz
zcat: bonds_201309.txt.gz.gz: No such file or directory
bonds_201310.txt.gz
zcat: bonds_201310.txt.gz.gz: No such file or directory
bonds_201311.txt.gz
zcat: bonds_201311.txt.gz.gz: No such file or directory
bonds_201312.txt.gz
zcat: bonds_201312.txt.gz.gz: No such file or directory

Seems like zcat is expanding the filename with an extra .gz However its not doing it when I try to zcat a file from the command line just picks up the filename which is provided and doing its job. Why is it happening when called in a loop?

I know I can achieve the same by using zgrep or find, but still interested in understanding this behaviour of zcat. Thanks!

Solution

  • I tried your script, there is no issue, but if I set alias ls='ls --color=always', I will have the same issue

    maybe your ls returns some hidden characters like color code
    can you try 

    for f in `\ls *.txt.gz`

    but you could have used

    for f in *.txt.gz instead of 

    for f in `ls *.txt.gz`

    and if your system has zgrep, you could use zgrep MYBOND $f instead of zcat $f | grep MYBOND

    The code will be like this:

    for f in *.txt.gz; do echo "$f"; zgrep MYBOND "$f"; done