Search code examples
dockerapache-kafkafluentd

fluent plugin kafka json escaping

I am using version fluent-plugin-kafka version 0.12.3.

I have an application that outputs its logs in JSON format, but my console consumer shows that the logs are escaped.

for instance, the application output the following log line:

{
  "msg": "ok"
}

when i look at the log using the console consumer, it outputs:

{
  "container_id": "7e...",
  "container_name": "/app",
  "source": "stdout",
  "log": "{\"msg\": \"ok\"}"
}

how should fluent-plugin-kafka should be configured so the application log will be nested as a json without escaping it?

desired output:

{
  "container_id": "7e...",
  "container_name": "/app",
  "source": "stdout",
  "log": {
    "msg": "ok"
  }
}
Solution

  • using record transformer filter i was able to mutate the incoming data.