Is it possible to create custom fields in a Kibana dashboard?

Question

I am using a Java micro-service architecture in my application and generating separate log files for each micro-service.

I am using ELK stack approach to visualize the logs in Kibana, but the problem is whatever the fields that I'm getting from Elastic Search that are related to server logs fields. some example fields are @timestamp,@version,@path,@version.keyword,@host.

i want to customize this fields by adding some fields like customerId,txn-Id,mobile no so that we can analyze the data easily.

I'm using org.apache.logging.log4j2 to write the logs. Can I set above fields (customerId,txn-Id,mobile) to log files? And then Elastic will store these fields with the above default fields and then these custom fields should available in a Kibana dashboard. Is this possible?

Answer 1

It's definitely possible to do that. I've not done it with the log4j2 stack (I have with slf4j/logback), but the basic approach is:

• set those fields in the Mapped Diagnostic Context (I'm fairly sure log4j2 supports that)
• use a log appender which logs to logstash-structured JSON
• configure filebeat to ship the JSON logs
• if filebeat is shipping to logstash, you'll need to configure logstash to pass those preformatted JSON logs directly to elasticsearch