writing a filename after each processed line

Question

I wrote following codes to process each line from multiple files and add filename at the end of the line:

for i in $(cat all_output_file.txt);do cat $i|sed 's/:/ /g'|sed 's/-/ /g'| sed 's/,cn=/ /g'| awk 'BEGIN{OFS="\t"}{print $1,$2,$3,$6,FILENAME}'>>input_file.txt;done

however, above code printed "-" at the end of every line instead of filename, how can I print filename at the end of each processed line?

Answer 1

It is because awk is reading from stdin not from a file.

Having said that you can do all this in awk itself:

while IFS= read -r line; do
    awk 'BEGIN{OFS="\t"}{gsub(/[:-]|,cn/, " "); print $1,$2,$3,$6, FILENAME}' "$line" 
done < all_output_file.txt >>input_file.txt

gsub is equivalent of 3 sed commands.

Also check this BASH FAQ on reading a file line by line

As @123 commented below, you can even avoid the for loop and use:

awk -v OFS='\t' '{gsub(/[:-]|,cn/, " ");
    print $1,$2,$3,$6, FILENAME}' $(<all_output_file.txt) >>input_file.txt

In BASH $(<all_output_file.txt) reads the content of input file.