Search code examples
regexlogstashlogstash-grok

How to use a grok to extract Service url component out of log


I'm trying to use a grok expression to extract the service url and time out of the expression posted below, but because there's multiple urls - my solution often retrieves the wrong url - so its not really consistent.

I've tried %{URIPATH:Path1}%{SPACE}%{URIPATH:ServiceURI}%{SYSLOGTIMESTAMP:time}

This doesn't work at all , but if I remove the {SYSLOGTIMESTAMP:time} - it gives me the result I'm looking for but it's not consistent with the other logs as they have a different format. So I'm trying to find a consistent way of getting the time, and Service URL out of a log.

Jun 12 04:27:35 1560306455 INCOMING: information 22.244.42.41 Jun 12 04:27:22 DPPRD01 [host_services][0x80e0013a][mpgw][info] source-https(IMS_SSL_29982): trans(2797190703)[12.6.1.16]: Received HTTP/1.1 POST for /services/NHgetInternetLimitsV1 from 10.6.17.166

What I expect is something like

time : Jun 12 04:27:35 service : NHgetInternetLimitsV1 or /services/NHgetInternetLimitsV1


Solution

  • You may use

    %{SYSLOGTIMESTAMP:time}.*POST for %{URIPATH:ServiceURI}
    

    It will extract

    {
      "time": [
        [
          "Jun 12 04:27:35"
        ]
      ],
      "ServiceURI": [
        [
          "/services/NHgetInternetLimitsV1"
        ]
      ]
    }
    

    Note that due to the .*POST for part, the last Service URI after POST for substring will be matched.