Search code examples
elasticsearchlogstashgsublogstash-grokgrok

How to remove part of the string before specific word using grok or gsub in logstash?


I have a string field "origin_message". It is pretty big one (used multiline to get mail content. Example of "origin_message":

Delivered-to: [email protected] A LOT OF OTHER CONTENT Subject: Subject goes here AND THE REST OF THE MESSAGE

Desired result:

Subject goes here AND THE REST OF THE MESSAGE

Is there a way to trim everything before "Subject:" phrase?

I have tried the following filter with no luck:

filter {
mutate {
    add_field => { "original_message" => "%{message}" }
    convert => {
        "original_message" => "string"
    }

    gsub => [
        "original_message", "^(.*)Subject", " "
    ]
}
}

Solution

  • No sure why but using gsub on "message" field before copying that to separate "original_message" field fixed the issue.

    filter {
    mutate {
        gsub => ["message", "^(.*)Subject", " "]
        add_field => { "original_message" => "%{message}" }
    
        convert => {
            "original_message" => "string"
        }
    }
    }
    

    @Val, thanks for verification. Issue appeared to be not pattern related.