Kibana Dashboard: Aggregation on Partial Message Match

Question

I'm working on a service that allows users to complete and submit several forms. After each form is submitted a message is logged like:

Form submitted Form-001 with draftId (unique Id) and submissionRef (unique ref)

I want to aggregate (and count) and visualise how many of each form (based on form name Form-xxx) is submitted.

I'm fairly new to kibana dashboards. So, all suggestions are appreciated.

Answer 1

I finally figured out how to achieve this. I'm posting it here for the purpose of closure and future references:

The way to do this is with a scripted field in Kibana. A scripted field can read messages and aggregate them as required. Kibana accepts scripted fields in 2 different languages: Lucene expressions and Painless.

My solution is written in painless:

def msg = doc['message.raw'].value;
if(msg != null){ 
    int flagIndex = msg.indexOf('Form submitted');
    if(flagIndex>0){ 
        int toIndex = msg.indexOf(' with draftId');
        return msg.substring(flagIndex+14,toIndex);
    }
}
return "";