Search code examples
fluentd

Is it possible to prepend and append to fluentd events?


Currently using fluentd to stream a third party's application logs to stdout.

Log that is received by fluentd is:

Jun 12, 2020 11:40:00 PM UTC INFO    [com.app.purge.PurgeManager run] PURGE: appAtom purge local data complete

Essentially, I want to be able to manipulate this log entry to become:

[LOG_START] [APP_LOG] Jun 12, 2020 11:40:00 PM UTC INFO    [com.app.purge.PurgeManager run] PURGE: appAtom purge local data complete [LOG_END]

Went through a lot of the plugins in the fluentd documentation but couldn't find anything that does this.

Fluentd configuration:

<source>
  @type tail
  path "path/Molecule/logs/*.shared_http_server.deployment.log"
  pos_file "path/fluentd/access.pos"
  tag app.access
  read_from_head true
  refresh_interval 1s
  <parse>
    @type none
  </parse>
</source>

<match app.access>
  @type stdout
  <format>
    @type single_value
  </format>
</match>

Any help would be appreciated. Thanks.


Solution

  • You can use fluentd record_transformer plugin to append any string to your log record. Quoting this link from fluentd docs.

    <filter foo.bar>
      @type record_transformer
      <record>
        message yay, ${record["message"]}
      </record>
    </filter>
    

    An input like {"message":"hello world!"} is transformed into {"message":"yay, hello world!"}