I have a sequence data file in fastq format https://en.wikipedia.org/wiki/FASTQ_format where the first line is sequence ID, the second line is sequence [ACGT], 3rd line is '+' and 4th line is quality values.
@M01610:118:000000000-D49F3:1:1101:14523:2546 1:N:0:CTTGTA
GTACACCTTCATGAAGAACTCCATCACCTTCATCTCCAGGATGCGGTCCTGGGTGCTGTTCCTGGCGATCTCGATCAGCTCGATGTACTCGTGGGGCACGTACTTCAGCTTGTGCCGCAGCTCGGACTTCTTCTCCTCCAGCTCGCTCTTCACCAGCTGGGATCCCCGCAGGTGTATCTTGGTATGCTTGTTCAGGTTGGAGCGGTGGGCAAATTTCCTCCCACAAATGTCACAGGCAAAAGGCTTCTC
+
CCCCCFFFFFFFGGGGGGGGGGHHHHHHHHHHHHHHGHHHGHHHGGGGGHHHGFGGHHHHHHHFHHGGGGHHHGGHGHHHHGGHHHHHHHGHGGGGGGGGHHHHHHHGHHHHHHHHGGGGGGHGGGGGHHHHHHHHHHHHHHHHGGGHHHHHHHHFHHHGEHFGHHGGGGGGGHGHFHHHHHFFHHGGGGGGGGGGFFF?FGGGGFGGGFFFFFFFFFFFFEFFF?FFFFFFEFFEFFFFBFFFFFBFF
@M01610:118:000000000-D49F3:1:1101:9569:5713 1:N:0:CTTGTA
CAAGGAAGGCACGGGGGAGGGGCAAACAACAGATGGCTGGCAACTAGAAGGCACAGGCTAGCCAGGCGGGGAGGCGGCCCAAAGGGAGATCCGACTCGTCGGAGGCCGAAAGCGAAGACGCGGGAGAGGCCGCAGAACCGGCAGAAGGCCTCGGGAAGGGAGGTCCGCTGGATTGAGAGCCGAAGGGACGTAGCAGAAGGACGTCCCGCGCAGGATCCAGTTGGCAACACAGGCGAGCAGCCAAGG
+
CDCCDFFFDCFFGGGGGGGGGGGGGHHHHHHHHHHHHHGGGHHHHHHHHHHHHHHHHGHHHHHHHGHGGGGGCGGFGGGGDHHHHGHGGHHHHGGGGFHGFGAGGGGGAAGFFDBF-DFFF>DF;DFAFDF=CA>CFBE>FFCFEFBFFF0FDDFAFFFFEDC.BFFFDBF.FFEBFFFEFAAC=FFE?>AEFEBFBFFFFFFDFFFFC>-9>=ABFFFFBFFFFFFFFFEFFFCFFA9BBEAFEF
I want to remove all the entries where any character in 4th line does not match either of the symbols ?@ABCDEFGHIJK
The output of the above example will be
@M01610:118:000000000-D49F3:1:1101:14523:2546 1:N:0:CTTGTA
GTACACCTTCATGAAGAACTCCATCACCTTCATCTCCAGGATGCGGTCCTGGGTGCTGTTCCTGGCGATCTCGATCAGCTCGATGTACTCGTGGGGCACGTACTTCAGCTTGTGCCGCAGCTCGGACTTCTTCTCCTCCAGCTCGCTCTTCACCAGCTGGGATCCCCGCAGGTGTATCTTGGTATGCTTGTTCAGGTTGGAGCGGTGGGCAAATTTCCTCCCACAAATGTCACAGGCAAAAGGCTTCTC
+
CCCCCFFFFFFFGGGGGGGGGGHHHHHHHHHHHHHHGHHHGHHHGGGGGHHHGFGGHHHHHHHFHHGGGGHHHGGHGHHHHGGHHHHHHHGHGGGGGGGGHHHHHHHGHHHHHHHHGGGGGGHGGGGGHHHHHHHHHHHHHHHHGGGHHHHHHHHFHHHGEHFGHHGGGGGGGHGHFHHHHHFFHHGGGGGGGGGGFFF?FGGGGFGGGFFFFFFFFFFFFEFFF?FFFFFFEFFEFFFFBFFFFFBFF
Here, the 4th line of the seq ID @M01610:118:000000000-D49F3:1:1101:9569:5713 1:N:0:CTTGTA contains symbols other than ?@ABCDEFGHIJK and is therefore removed.
The length of the 2nd and 4th line for a seq ID is the same but it is different for different seqID ranging from (200 to 250).
A unit is made of 4 lines (seq ID, sequence, + sing, quality for sequence). I want to remove all units where quality of sequence (4th line) for each character in the sequence (2nd line) matches anything other than either characters of the pattern ?@ABCDEFGHIJK. I have tried this code and still working on it
cat file.fq | awk 'NR%4==0' | xargs -n1 awk '{ for(i=0; ++i <= length($0);) printf "%s\n" }'
I will appreciate any help.
$ awk '{unit=unit $0 ORS} NR%4==0{if (/^[?@ABCDEFGHIJK]+$/) printf "%s", unit; unit=""}' file
@M01610:118:000000000-D49F3:1:1101:14523:2546 1:N:0:CTTGTA
GTACACCTTCATGAAGAACTCCATCACCTTCATCTCCAGGATGCGGTCCTGGGTGCTGTTCCTGGCGATCTCGATCAGCTCGATGTACTCGTGGGGCACGTACTTCAGCTTGTGCCGCAGCTCGGACTTCTTCTCCTCCAGCTCGCTCTTCACCAGCTGGGATCCCCGCAGGTGTATCTTGGTATGCTTGTTCAGGTTGGAGCGGTGGGCAAATTTCCTCCCACAAATGTCACAGGCAAAAGGCTTCTC
+
CCCCCFFFFFFFGGGGGGGGGGHHHHHHHHHHHHHHGHHHGHHHGGGGGHHHGFGGHHHHHHHFHHGGGGHHHGGHGHHHHGGHHHHHHHGHGGGGGGGGHHHHHHHGHHHHHHHHGGGGGGHGGGGGHHHHHHHHHHHHHHHHGGGHHHHHHHHFHHHGEHFGHHGGGGGGGHGHFHHHHHFFHHGGGGGGGGGGFFF?FGGGGFGGGFFFFFFFFFFFFEFFF?FFFFFFEFFEFFFFBFFFFFBFF