perl script to remove next line with duplicate words

Question

input:

DFF_2 : dff_0_2 port map(READY_c => READY_c, CT0 =>CT0);
\DFF_0\ : dff_0 port map(un1_CT1 => un1_CT1, CT2 =>CT2);
DFF_10 : dff_0_10 port map(MRVQN1 => MRVQN1, un1_CT2_1 =>GSMC_un1_CT2_1);
DFF_1 : dff_0_1 port map(un1_CT2_1 =>GSMC_un1_CT2_1);
DFF_1 : dff_0_1 port map(un1_CT2_1 =>un1_CT2_1);

expected output1:

DFF_2 : dff_0_2 port map(READY_c => READY_c, CT0 =>CT0);
\DFF_0\ : dff_0 port map(un1_CT1 => un1_CT1, CT2 =>CT2);
DFF_10 : dff_0_10 port map(MRVQN1 => MRVQN1, un1_CT2_1 =>GSMC_un1_CT2_1);
DFF_1 : dff_0_1 port map(un1_CT2_1 =>un1_CT2_1);

expected output2: (no need to be in order, but updated line should be resumed)

DFF_1 : dff_0_1 port map(un1_CT2_1 =>un1_CT2_1);    
DFF_10 : dff_0_10 port map(MRVQN1 => MRVQN1, un1_CT2_1 =>GSMC_un1_CT2_1);
\DFF_0\ : dff_0 port map(un1_CT1 => un1_CT1, CT2 =>CT2);    
DFF_2 : dff_0_2 port map(READY_c => READY_c, CT0 =>CT0);

I can't use duplicate line removal perl script for this situation as the string word8 is updated with new string word10. I tried like reverse the content and apply line with duplicate word to be removed.but, could not achieve it by my code.

open (IN, "<input.txt") or die;
open (OUT, ">output.txt") or die;
my @reverse = reverse <IN>;
foreach (@reverse){
print OUT "@reverse\n"; }
close (IN);
close (OUT);  

output:

DFF_1 : dff_0_1 port map(un1_CT2_1 =>un1_CT2_1);    
DFF_1 : dff_0_1 port map(un1_CT2_1 =>GSMC_un1_CT2_1);
DFF_10 : dff_0_10 port map(MRVQN1 => MRVQN1, un1_CT2_1 =>GSMC_un1_CT2_1);
\DFF_0\ : dff_0 port map(un1_CT1 => un1_CT1, CT2 =>CT2);    
DFF_2 : dff_0_2 port map(READY_c => READY_c, CT0 =>CT0);




open (IN1, "<output.txt") or die;
open (OUT1, ">output1.txt") or die;
while (<IN1>){
my $save = "$1" if /(.+)\s\:/;
next if /$save\s/;
print OUT1 $_;}
close (IN1);
close (OUT1;

But it is not generating the output file as expected. Kindly help me out.

Answer 1

Use hash for to do it.

While iterate the loop try to split the line by using the :, So split the line with pattern matching like this ^.+?\K\s:

^ for the beginning of the match

+? helps to avoid the greed of the +.

\K for keep the word from splitting.

Then store the two data in $first and $second. The create the hash key by $first values. It helps to remove the duplicate. Then finally the uniq values is stored into the %hash then format the hash by using grep.

open my $fh,"<","one.txt";
my %hash;
while (<$fh>)
{   
    ($first,$second) = split(/^.+?\K\s:/);
    $hash{$first} = " : $second";

}

my @ar = grep{ $_ =$_.$hash{$_} }keys %hash;
print @ar;