Search code examples
linuxloopsfindfilenamescut

My loop is outputting to one concatenated file instead of several individual files

I have a bunch of tsv files each with 7 columns, but I am only interested in columns 1 and 7. Each file has the format: SampleName.bam.S.txt.

Example: 7805.bam.S.txt 7806.bam.S.txt 7808.bam.S.txt etc...

I've tried two things:

1) find . -type f -name '*.S.txt' -exec cut -f 1,7 {} > {}.F \; and
2) for f in '*.S.txt';do cut -f 1,7 "$f" > "$f".F;done

What I want is my directory to now be

7805.bam.S.txt 7805.bam.S.txt.F 7806.bam.S.txt 7806.bam.S.txt.F 7808.bam.S.txt 7808.bam.S.txt.F etc...

but instead I just get

1) 7805.bam.S.txt 7806.bam.S.txt 7808.bam.S.txt etc... {}.F
2) 7805.bam.S.txt 7806.bam.S.txt 7808.bam.S.txt etc... $f.F

Where the generated file has all of the outputs written to it, but how can I get each iteration to generate a unique filename? Thanks.

Solution

  • If I understand you correctly, this is how I would do it. Agree with previous answer about awk to deal with tsv/csv -- that's definitely the right tool. I just find bash syntax for iteration and variables easier to remember than awk syntax.

    find . -type f -name "*.S.txt" | while read FILE;do awk -F"\t" '{print $1"\t"$7}' $FILE > $FILE.F;done