Logstash extracting and customizing field with grok and ruby

Question

i have this data in elastic search logs saved in a referer field

/clientReq?sessionid=3332&UID=ed91b-517234-4f4c211-a20e-d2e1aefc126a&signUp=false

i want to use ruby to save this data ed91b-517234-4f4c211-a20e-d2e1aefc126a in a separate field.

i have tried this in ruby in my pattern configuration file,

ruby {
 code => "
    saveid=event[referer].match((\w+[-]?)+)+)
    event.set('saved',saveid)   "
}

this doesn't even save the entire filed. So i went ahead to try grok filter instead and tried this,

grok {
    match => {"message" => "%{COMBINEDAPACHELOG}"}
    add_field => { "savedData" => "%{referer}" }
}

neither of these works. I have tested configuration and if configuring successfully. when i visit kibana front end i don't see new field created either.

Answer 1

Ruby hash syntax event[field] = foo is not used anymore, and has been replaced by Get API for example, event.get(referrer).

Beside that, your regex is not correct to get desired results. One of the solutions is to use Positive Lookbehind to check for UID,

this should work,

ruby {
 code => "
    saveid = event.get('referer').match(/(?<=UID=)((\w+[-]?)+)+/)[1]
    event.set('saved',saveid)
    "
}

for grok, you can create a new filter for your referer field, and use the gork's predefined UUID pattern to match your string...can you try this,

grok {
  match => {"referer" => "UID=%{UUID:saveData}"}
}

hope this helps.