Search code examples
bashshellnagiosnrpe

How find duplicated[s] line[s] and remove it?


I had to append in my nrpe.cfg file this line, in more than 200 servers:

command[check_utc]=/company/nrpe/libexec/check_utc.sh

But for some issues some machines have more than once this line. So now I would like to check if there's more than once and remove it, leaving just one line with the specified command. I need to do this in shell script.


Solution

  • Method 1: Eliminate all duplicate lines

        cat filename | sort | uniq > outfile
        mv outfile filename
    

    The | characters feed the output of the commands on the left into the commands on the right as input.

    The cat command reads the contents of the file to standard output (the screen or another file)

    The sort command sorts the output alphabetically, which is needed before using uniq...

    The uniq command eliminates adjacent duplicate values

    The > outfile writes the output of the last step to a file named "outfile"

    The mv command replaces the original file with the new data file outfile

    This method will be destructive because it will remove all duplicate lines of ANY kind, not just the one you want to remove.

    Method 2: Remove all instances of the specific line, then re-add once

        cat filename | sed 's/command\[check_utc\]\=\/company\/nrpe\/libexec\/check_utc.sh//' > outfile
        mv outfile filename
    

    The sed command allows string substitutions to be performed on specific string patterns. the format for such a command is sed 's/find/replace/' to find the first instance of 'find' on each line and replace it with 'replace'. To make it work on every instance on each line you add 'g' to the end, i.e. sed 's/find/replace/g'

    The \ characters cause sed to literally interpret special characters in the string that would otherwise be misread as special instructions.

    The command above will completely remove all instances of that specific string (replacing it with nothing), which you will then need to follow with editing the code and adding it back in once

    Method 3: Remove all instances, then re-add automatically to the end of the file

        cat filename | sed 's/command\[check_utc\]\=\/company\/nrpe\/libexec\/check_utc.sh//' > outfile
        mv outfile filename
        echo "command[check_utc]=/company/nrpe/libexec/check_utc.sh" >> filename
    

    This is the same code as above, but after removing all instances of the string, you use echo to print the line and use >> to append the text to the end of the contents of filename