Search code examples
linuxsortingterminaltext-processingdata-processing

How to remove repeated lines but not sorting (keep the initial order)

I have this input:

Host: ping.chartbeat.net
Host: extra.test.co
userblablabla
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: extra.test.co
Host: secure.test.com
userlalalalala
Host: ping.chartbeat.net
Host: secure.test.com
Host: secure.test.com
Host: secure.test.com

and I want to convert it into this:

Host: ping.chartbeat.net
Host: extra.test.co
userblablabla
Host: secure.test.com
userlalalalala

How do I perform that¿?

I tried |sort -u but that breaks the order in which I want the lines

Solution

  • You want to use uniq <file>. uniq is part of the GNU coreutils.