fluentd JSON log field not being parsed

Question

I'm following the fluentd tutorial at https://docs.fluentd.org/container-deployment/docker-logging-driver

But I'm unable to make the JSON parser work.

I'm running fluentd as follow:

# ./demo.conf
<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *>
  @type stdout
</match>

<filter docker.**>
  @type parser
  key_name log
  reserve_data true
  <parse>
    @type json # apache2, nginx, etc...
  </parse>
</filter>

docker run -it -p 24224:24224 -v $(pwd)/demo.conf:/fluentd/etc/demo.conf -e FLUENTD_CONF=demo.conf fluent/fluentd:v1.3.2

and I'm using the following docker run command to mock a JSON log to fluentd

docker run --log-driver=fluentd ubuntu echo '{"test":"test"}'

Expected: 2020-06-23 13:52:31.000000000 +0000 e54806b99130: {"log":"{\"test\":\"test\"}","container_id":"e54806b9913010d5ecb79b305d227db9e706299ad94bd070be6042dce735ed3b","container_name":"/wizardly_kalam","source":"stdout","test":"test"}
Actual:   2020-06-23 13:52:31.000000000 +0000 e54806b99130: {"log":"{\"test\":\"test\"}","container_id":"e54806b9913010d5ecb79b305d227db9e706299ad94bd070be6042dce735ed3b","container_name":"/wizardly_kalam","source":"stdout"}

Acording to the reserve_data doc it suppose to work but I'm unable to understand what is the issue.

Answer 1

Maybe I'm missing something, but doesn't Fluentd respect the order of steps in the config? You print to stdout before parsing the fields. Try this:

# ./demo.conf
<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<filter docker.**>
  @type parser
  key_name log
  reserve_data true
  <parse>
    @type json # apache2, nginx, etc...
  </parse>
</filter>

<match *>
  @type stdout
</match>