Search code examples
bashsedappendconcatenationprepend

Concatenating a portion of each line of a file using BASH commands

BASH version in use:

echo $BASH_VERSION
4.4.19(1)-release

I am using the following plumbing to extract all banned IP addresses from a fail2ban log file:

cat /var/www/html/fail2ban.log | grep Ban | cut -d "]" -f3 | cut -d "r" -f2 | cut -d "e" -f2 | cut -d " " -f3 | uq | sort -n > /var/www/html/fail2ban.html

This results in a HTML file with a list of IP addresses ordered from low to high.

My goal is to make each of the IP addresses clickable hyperlinks, for usability reasons. I have been successful using the following operations:

Step 1 - Prepend each line with the beginning portion of HTML hyperlinks.

Code used (successful): sed -e 's/^/<a href=\"http:\/\//' -i /var/www/html/fail2ban.html

Step 2 - Append each line with the closing portion of HTML hyperlinks and a line break.

Code used (successful): sed -e 's/$/\">CLICK HERE<\/a><\/br>/' -i /var/www/html/fail2ban.html

Result - The entire HTML file now contains a list of links such as the following:

<a href="http://10.30.20.40">CLICK HERE</a></br>

Problem - Rather than "CLICK HERE" as the link text, I want to display the corresponding IP address instead. Example of desired result using example above:

<a href="http://10.30.20.40">http://10.30.20.40</a></br>

How can this final step be achieved using sed, awk, etc.?

Example input from file /var/www/html/fail2ban.html file containing the following three lines:

<a href="http://1.2.3.4">CLICK HERE</a></br>
<a href="http://5.6.7.8">CLICK HERE</a></br>
<a href="http://9.10.11.12">CLICK HERE</a></br>

Desired output:

<a href="http://1.2.3.4">http://1.2.3.4</a></br> 
<a href="http://5.6.7.8">http://5.6.7.8</a></br>
<a href="http://9.10.11.12">http://9.10.11.12</a></br>
Solution

  • You may use a single sed command:

    sed -e 's,^.*$,<a href="http://&">&</a></br>,' -i /var/www/html/fail2ban.html

    See the online demo:

    s="10.23.34.23
56.67.22.12"
sed -e 's,^.*$,<a href="http://&">&</a></br>,' <<< "$s"

    Output:

    <a href="http://10.23.34.23">10.23.34.23</a></br>
<a href="http://56.67.22.12">56.67.22.12</a></br>

    Notes:

    • ^.*$ matches any line in full
    • The & in the replacement refers to the full line text matches with .*
    • The , char is used instead of / to avoid overescaping / chars in the regex and replacement patterns.