Search code examples
apache-kafkafilebeat

What does %{[offset]} mean in codec.format?

I have a question about filebeat.yml file.

In my filebit.yml setting, input is log and output is kafka.

When used as below, does the meaning of %{[offset]} mean offset of the log file?

filebeat.inputs:
  - type: log
     ...
     paths:
      - /home/log/test.log
     ...

output.kafka:
  codec.format: 
    string: '"offset":%{[offset]}, "message":%{[message]}'

Additionally, %{[offset]} What is the meaning of using %{[kafka.offset]} instead?

Does it mean the offset of the matching kafka partition?

Solution

  • Filebeat's log input type adds an offset metadata field to the event. It's not the Kafka offset that the record gets put at