Search code examples
elasticsearchlogstashkibanametricbeat

How can I parse custom Metricbeat dictionary for Kibana?

I have logstash set up sending to Kibana, and it tags each log file with two custom fields -- Cluster and Node: enter image description here

I would like to add the same two fields to my Metricbeat configuration using the "fields" option, but when I do this it comes through as a dictionary in Kibana: enter image description here

Here is the Metricbeat config file I'm using:

metricbeat.modules:
- module: system
  metricsets:
    # CPU stats
    - cpu

    # System Load stats
    - load

    # Per CPU core stats
    #- core

    # IO stats
    #- diskio

    # Per filesystem stats
    - filesystem

    # File system summary stats
    - fsstat

    # Memory stats
    - memory

    # Network stats
    - network

    # Per process stats
    - process

    # Sockets (linux only)
    #- socket
  enabled: true
  period: 10s
  processes: ['.*']

  #Add Supervisor Cluster and Node ID
  fields:
    cluster: "Upswell ELK Testing Cluster"
    node: "Nina's Macbook"

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:

  # Array of hosts to connect to.
  hosts: ["http://localhost:9200"]

I can't find any documentation about how to format these fields; is this an adjustment I should be making in Logstash or in the Metricbeat configuration?

Solution

  • From : https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-general.html

    fields : By default, the fields that you specify here will be grouped under a fields sub-dictionary in the output document. To store the custom fields as top-level fields, set the fields_under_root option to true.