Fluentd input_tail plugin generates different event key_name

Question

I was to told collect 3 JAVA Logs to ES, the log have logrotate.

I'm using in_tail plugin to collect Logs. I'm using td-agent v4, installed by yum

I notice that td-agent have some contradictory error logs：

2022-07-15 09:58:32 +0800 [warn]: #0 dump an error event: error_class=ArgumentError error="message does not exist" location=nil tag="condition-service" time=2022-07-15 09:57:26.032947914 +0800 record={"Log"=>"2022-07-15 09:57:26.032 [DubboServerHandler-10.65.8.13:20882-thread-100] ..."}

... I notice error above and then modify key_name

2022-07-15 16:44:37 +0800 [warn]: #0 dump an error event: error_class=ArgumentError error="Log does not exist" location=nil tag="condition-service" time=2022-07-15 16:44:37.938689464 +0800 record={"message"=>"2022-07-15 16:44:37.938 [checkQuoteAutoPush-thread] ..."}

I think it is the in_tail generates differnet event key_name, why? How do I control key_name and fix it?

Here is my td-agent configuration:

<system>
    log_level info
</system>

<source>
@type tail
@label @condition-service
path /XXXX/log1.log,/XXXX/log2.log
pos_file /var/log/td-agent/tmp/condition-service.log.pos
tag condition-service
pos_file_compaction_interval 24h
<parse>
    @type none
</parse>
</source>

<source>
@type tail
@label @condition-quotes
path /XXXX/log3.log
pos_file /var/log/td-agent/tmp/condition-quotes.log.pos
tag condition-quotes
pos_file_compaction_interval 24h
<parse>
    @type none
</parse>
</source>

<label @condition-quotes>
    <filter condition-quotes>
        @type parser
        key_name Log
        <parse>
            @type regexp
            expression /^(?<Log>.*)/gm
        </parse>
    </filter>

    <match condition-quotes>
        @type elasticsearch
        host  XX.XX.XX.XX
        port 9200
        logstash_format true
        logstash_prefix ${tag}
    </match>
</label>

<label @condition-service>
    <filter condition-service>
        @type parser
        key_name Log
        <parse>
            @type regexp
            expression /^(?<Log>.*)/gm
        </parse>
    </filter>

    <match condition-service>
        @type elasticsearch
        host XX.XX.XX.XX
        port 9200
        logstash_format true
        logstash_prefix ${tag}
    </match>
</label>

Answer 1

After I review my configuration, I think the root cause is none parser plugin, I should specify message_key param.

...
<parse>
    @type none
    message_key Log
</parse>
...