Search code examples
logstashlogstash-grok

grok filter to extract some message from two specific words

sample log file is below 

2018-07-02 09:35:57 991 [INFO] from application in pool-2-thread-9 - Authenticate document processing time for transactionId : 1271400374895007_node1 documentType : Passport is 1629 msec

I wrote grok filter to extract some fields like transaction, document type, duration 

%{TIMESTAMP_ISO8601:timestamp} (\[%{WORD:loglevel}\]) (?<logger>(?:[a-zA-Z0-9-]+\.)*[A-Za-z0-9$]+)\s+(-\s+)? %{GREEDYDATA} .*transactionId : %{WORD:transactionid} documentType : %{WORD:document type} is (?<duration>.*msec

can someone please suggest how to extract data between two specific words "-" (between-message) "processing time"

Solution

  • you can create a custom pattern to match everything between - and processing time,

    (?<pool_thread>\w+[-]\d+[-]\w+[-]\d+\s*?)-(?<custom_word>.*?)(processing time)

    This will output,

    {
  "pool_thread": [
    [
      "pool-2-thread-9 "
    ]
  ],
  "custom_word": [
    [
      " Authenticate document "
    ]
  ]
}