Search code examples
amazon-web-servicesnginxelasticsearchfluentfluentd

fluentd doesn't tag the events properly


I have a simple fluentd configuration:

<source>
  type forward
</source>

<source>
  type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nignx-access.log
  tag nginx.access
  format nginx
</source>

<source>
  type tail
  path /var/log/nginx/error.log
  pos_file /var/log/td-agent/nginx-error.log
  tag nginx.error
  format nginx
</source>

And then I have the appropriate <match> tag after the sources. These logs are being shipped to an ElasticSearch instance on AWS. The problem is that, they show up but without a tag. So the nginx access log shows up properly but with not 'nginx.access' tag. This causes problems because now there's no way to categorize the logs.


Solution

  • Credit to @repeatedly on fluentd's slack:

    include this in the match tag :

    <match my.logs>
      @type elasticsearch
      include_tag_key true
      tag_key _key
    </match>