Search code examples
elasticsearchloggingfluentd

How to add extra data to the messages going to elasticsearch in fluentd


I have setup elasticsearch and fluentd on my machine. I have written a basic configuration file which is monitoring a log file and sending the logs to elasticsearch. Below is the configuration file:

<source>
  @type tail
  path /home/user/log.json
  pos_file /home/user/log.json.pos
  format json
  time_format %Y-%m-%d %H:%M:%S
  tag first
</source>

<match *first*>
  @type elasticsearch
  hosts 192.168.60.118:9200
  user <username>
  password <password>
</match>

Below is the log message received on elasticsearch:

{
  "_index": "fluentd",
  "_type": "fluentd",
  "_id": "2987",
  "_version": 88,
  "_score": null,
  "_source": {
    "DataNumber": "030",
    "DataId": "MMX56",
    "DataCount": 87,
    "Status": "Done"
  },
  "fields": {
     "Created": [
      "2018-06-11T05:27:20.278Z"
     ]
  },
  "sort": [
    1528694840278
  ]
}

With the above message, I also want to add other information like the machine number, floor number which has generated this data and other stuff. I cannot hardcode this details in the code which is generating these logs. So I am looking for a way to add these details in the fluentd configuration file so that every outbound message to elasticsearch is appended with machine number & floor number.

Thanks


Solution

  • Take a look at record:

    <record>
        hostname "#{Socket.gethostname}"
        tag ${tag}
    </record>