Search code examples
powershellhashtable

Convert firewall log to Powershell hashtable


I have a firewall log of a Sophos UTM containing several entries that look something like the example at the end (each entry is a single line).

I want to write a Powershell script to do some statistics on how often a specific rule has matched and stuff like this. To perform these analyses I would like to convert each line of the log to a hashtable so I can access the information of the entries easily.

Is there any fancy way to do it instead of iterating over a string, that contains one line of the firewall log, and matching chars (e.g. search for '="' and then search to the left and right of this position to get the attributes and corresponding values) the to find the pairs of attribute an value?

Any help and ideas how to convert the data in a useable format will be appreciated!

2021:09:08-00:02:45 fwtest ulogd[4040]: id="2021" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60019" initf="eth2" srcmac="01:23:45:67:89:ab" dstmac="ba:98:76:54:32:10" srcip="10.0.0.1" dstip="10.0.1.1" proto="17" length="96" tos="0x00" prec="0x00" ttl="45" srcport="1234" dstport="4321"

Solution

  • Is there any fancy way to do it instead of iterating over a string

    Any way you do it will involve some form of reading and parsing the string. You can simplify the code needed by using regex to match the name="value" pairs:

    # Replace this string array with `Get-Content path\to\log\file(s).log` 
    @(
      '2021:09:08-00:02:45 fwtest ulogd[4040]: id="2021" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60019" initf="eth2" srcmac="01:23:45:67:89:ab" dstmac="ba:98:76:54:32:10" srcip="10.0.0.1" dstip="10.0.1.1" proto="17" length="96" tos="0x00" prec="0x00" ttl="45" srcport="1234" dstport="4321"'
    ) |Select-String -Pattern '\b(\w+)\="([^"]+)"' -AllMatches |ForEach-Object {
      # Create an ordered dictionary to hold the name-value pairs
      $props = [ordered]@{}
    
      # Iterate over each pair found by Select-String
      $_.Matches |ForEach-Object {
        # Add each name-value pair to the dictionary
        $props[$_.Groups[1].Value] = $_.Groups[2].Value
      }
    
      # Convert dictionary to PSObject
      [pscustomobject]$props
    }
    

    This will produce 1 new object per log line that you can work with using builtin utilities like Where-Object, Group-Object, Measure-Object etc.

    The regex pattern used (\b(\w+)\="([^"]+)") describes:

    \b          # match a word boundary
      (         # capture group start
       \w+      # match 1 or more word characters (letters or numbers)
      )         # capture group end
      \="       # match a literal = followed by "
      (         # capture group start
       [^"]+    # match 1 or more of any characters except "
      )         # capture group end
      "         # match a literal "