Search code examples
bashgrepnmap

How can I analyze multiple blobs (multi-lines) of text for patterns in a single file?


I have run an nmap scan using the --script ssl-enum-ciphers -p443 192.168.0.0/24 options against multiple IP addresses. I have also run an extremely similar scan using the --script ssh2-enum-algos -p22 options, that produces output in the same format.

I want to quickly analyze this data and zero in on specific matches of specific ciphers or algorithms. The overarching goal is to run an ad-hoc internal vulnerability assessment without access to fancy tools such as Nessus or Rapid7 InsightVM.

While nmap supports the -oX option to output to XML, I have found that neither Microsoft Word, Excel, or a web browser know how to open the file. The Microsoft products keep producing an error that the xml format is incorrect.

So then I tried the nmap-parse-output code on Github. While it will easily group IP addresses by ports, it doesn't appear to be able to take that a step further and analyze the ciphers or algorithms for me.

So now I'm trying to figure out a way to manually parse these blobs of data. A typical result might contain data for multiple IP addresses in the following format (the following example stdout is edited for brevity):

Nmap scan report for 192.168.1.1
Host is up (0.00064s latency).

PORT    STATE SERVICE  VERSION
443/tcp open  ssl/http lighttpd
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|     compressors: 
|       NULL
|     cipher preference: server
|_  least strength: A

Nmap scan report for 192.168.1.2
Host is up (0.00048s latency).

PORT    STATE SERVICE  VERSION
443/tcp open  ssl/http nginx (reverse proxy)
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
|     compressors: 
|       NULL
|     cipher preference: server
|_  least strength: A

Nmap scan report for 192.168.1.3
....

Nmap scan report for 192.168.1.4
....

How can write a script, using standard applications available to bash, to loop through each blob of text? We don't know the length of each blob, so I need to match on the string "Nmap scan report" or something similar, and extract the data that is in between each of those matches.

Something like this would get me started, but it isn't complete, and doesn't actually separate each blob individually:

for i in $(cat scan-results | grep "Nmap scan report for"); do more data analysis here; done 

For example, I might want to search for any IP address that support RC4 ciphers on port 443, so in the "do more data analysis", I would like to run: grep -i rc4

Or in the case of ssh algorithms, I want to ensure all cbc algorithms are disabled, so I could run: grep -i cbc

The resulting goal would be to list anything that matches so that I can quickly attribute the match to the specific IP address. I don't care how the results look, I just care about finding the results quickly.

Any help would be appreciated!


Solution

  • Without more sample data, and going solely on the limited examples, and keeping in mind that output format isn't of importance ...

    NOTE: My sample data file - nmap.dat - is a cut-n-paste copy of the sample nmap data provided by the OP.

    I'm thinking a multi-pattern grep may suffice, eg:

    # search for any IP address that support RC4 ciphers on port 443
    
    $ grep -i "Nmap scan report for|443|RC4" nmap.dat
    Nmap scan report for 192.168.1.1
    443/tcp open  ssl/http lighttpd
    Nmap scan report for 192.168.1.2
    443/tcp open  ssl/http nginx (reverse proxy)
    Nmap scan report for 192.168.1.3
    Nmap scan report for 192.168.1.4
    
    # want to ensure all cbc algorithms are disabled
    
    $ egrep -i "Nmap scan report for|cbc" nmap.dat
    Nmap scan report for 192.168.1.1
    Nmap scan report for 192.168.1.2
    |       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
    |       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
    Nmap scan report for 192.168.1.3
    Nmap scan report for 192.168.1.4