event_set (logstash ruby) does not give me a field?

Question

I may have misunderstood the documentation, but is not event_set supposed to give me a field that I can search for in (e.g.) kibana?

I've also looked at several posts, for which I think this is the closest to describing my problem(?). I have tried the exact syntax as suggested by @magnusbaeck in this post too, without succeeding in getting a new field defined this way.

I can't seem to get the event_set to work as I expect, and I cannot figure out how I'm meant to do this with the syntax introduced with version 5.0.

I grok two fields from my log file, named TimeSpentMinutes and TimeSpentSeconds, and I know there are several posts about this, but not so many that I've been able to find after the change with v.5.0.

Anyway, my log looks something like this (I've edited a bit to simplify):

1012 2017-10-27 05:03:24 [  2:31] some message blablabla

I then run the following trying to convert the TimeSpentMinutes into seconds and add these seconds to the valule in TimeSpentSeconds, then put them to a new field called TimeSpent:

filter {
  grok {
    match => {
      "message" => ["^%{NUMBER:LogLineID} %{TIMESTAMP_ISO8601:LogLineTime} \[%{SPACE}%{INT:TimeSpentMinutes}\:%{INT:TimeSpentSeconds}\] %{GREEDYDATA:LogText}$"]
    }
  }

  # Merge related minutes and seconds to one single value in seconds
  if [TimeSpentMinutes][TimeSpentMinutes] {
    ruby {
      code => 'event_set("TimeSpent", event_get("TimeSpentMinutes").to_f*60 + event_get("TimeSpentSeconds").to_f)'
    }
    mutate {
      remove_field => ['TimeSpentMinutes', 'TimeSpentSeconds']
    }
  }
}

From this I get:

LogLineID = 1012
LogLineTime = 2017-10-27 05:03:24
TimeSpentMinutes = 2
TimeSpentSeconds = 31
LogText = some message blablabla

So far so good, but it does not seem like the ruby filter does anything? At least, I do not get the field TimeSpent that I expect.

Do I need to add some add_field in a mutate or ruby filter? Or is there something else that I've misunderstood? I do not see any error messages, so it seems at least the syntax is more or less ok, but there must be something that's off or missing.

Answer 1

You somewhat misunderstood the documentation. The correct syntax is event.set (note the dot instead of a underscore)

See the example from the docs you linked, it demonstrates getting and setting fields:

filter {
  ruby {
    code => 'event.set("lowercase_field", event.get("message").downcase)'
  }
}

Your line of ruby code might look like this in the end:

ruby {
      code => 'event.set("TimeSpent", event.get("TimeSpentMinutes").to_f*60 + event.get("TimeSpentSeconds").to_f)'
    }