Logstash define field within a field

Question

Is there a way grok can save a field from within an already-defined field?

I have the following filter:

grok {
  match => [ "message", "\[%{TIMESTAMP_ISO8601:timestamp}\]\s+\[%{DATA:homebridge_app}\]\s+%{GREEDYDATA:log_message}" ]
}

Which results in "log_message" being populated with things like:

Current temperature for Entryway Thermostat is: 20.5 C
Current humidity for Entryway Thermostat is: 60%
Current temperature for Entryway Thermostat is: 21 C
Current temperature for Entryway Thermostat is: 21.5 C
Current humidity for Entryway Thermostat is: 61%

and so on. There can be other data in there, but the above are the lines I'm interested in.

I want to be able to use Kibana to plot humidity and temperature on charts against time, but I still want log_message to contain the full string as above. Is there a way I can save the integers from the log_message field, without splitting that field in two?

Full stack I'm using is Filebeat->Logstash->Elasticsearch->Kibana

Answer 1

You can add a second grok filter on the log_message field:

grok { 
    match => { "log_message" => [ 
        "%{NUMBER:humidity}%$", 
        "%{NUMBER:temperature} C$" 
    ] } }
}

See the grok plugin documentation:

If you need to match multiple patterns against a single field, the value can be an array of patterns

filter {
   grok {     
      match => { "message" => [ "Duration: %{NUMBER:duration}", "Speed: %{NUMBER:speed}" ] } 
   }
}