Search code examples
logstashelastic-stacklogstash-configuration

Logstash configuration for word extraction


I am new to Logstash manipulations and I have no idea how to do the below. I have a sample data as below:

Column:Type
Incident Response P3
Incident Resolution L1.5  P2
...

I want to extract the word 'Response' and 'Resolution' into a new column 'SLA type'

Im looking for something very alike to the below SQL statement:

case when Type like '%Resolution%' then Resolution
when Type like '%Response%' then Response 
end as SLA_Type

How do i manipulate this in Logstash?

Below is my conf. I'm using an API input.

input {

http_poller {
    urls => {
    snowinc => {
    url => "https://service-now.com"
    user => "your_user"
    password => "yourpassword"
    headers => {Accept => "application/json"}
    }
}
request_timeout => 60
metadata_target => "http_poller_metadata"
schedule => { cron => "* * * * * UTC"}
codec => "json"
}
}
filter
   {
   json {source => "result" }
   split{ field => ["result"] }
date {
  match => ["[result][sys_created_on]","yyyy-MM-dd HH:mm:ss"]
  target => "sys_created_on"
     }
}
output {
  elasticsearch {
    hosts => ["yourelastuicIP"]
    index => "incidentsnow"
    action=>update
    document_id => "%{[result][number]}"
    doc_as_upsert =>true
}
        stdout { codec => rubydebug }
}

The output for the API json url looks like the below:

{"result":[
{
"made_sla":"true",
"Type":"incident resolution p3",
"sys_updated_on":"2019-12-23 05:00:00",
"number":"INC0010275",
"category":"Network"} ,
{
"made_sla":"true",
"Type":"incident resolution l1.5 p4",
"sys_updated_on":"2019-12-24 07:00:00",
"number":"INC0010567",
"category":"DB"}]}

Solution

  • You can use the following filter block in your pipeline to add a new field if a word is present in another field.

    if "response" in [Type] {
        mutate {
            add_field => { "SLA_Type" => "Response" }
        }
    }
    if "resolution" in [Type] {
        mutate {
            add_field => { "SLA_Type" => "Resolution" }
        }
    }
    

    If the word response is present in the field Type a new field named SLA_Type with the value Response will be added to your document, the same in will happen with resolution.