I have a FluentBit container receiving data from a WebAPI via OpenTelemetry input. Using stdout output I see, that the log is added to FluentBit in the structured form (including attributes/properties/...whatever-you-want-to-call-it...), not only the pure message.
e.g.:
[2] opentelemetry.0: [[1717162605.894517831, {"EnvName"=>"Development", "{OriginalFormat}"=>"Hosting environment: {EnvName}"}], {"message"=>"Hosting environment: Development"}]
Not the EnvName
in the first [[
. Now I want that output to Loki target.
The fluent bit YAML I use:
pipeline:
inputs:
- name: opentelemetry
listen: 0.0.0.0
outputs:
- name: loki
match: '*'
host: host.docker.internal
port: 3100
labels: job=my
Loki is a different container on same host. The connection itself does work, but Loki only receives the last part, the {"message"=>"Hosting environment: Development"}
without the attributes. But I do know Loki is able to handle structured logs, I did it in a different project (without OTel in between).
Now the question is: How do I tell FluentBit to include the attributes it receives from OTel in the message it sends to Loki?
(Bonus point if someone can tell me which tags I need to use in match
for Loki output. Neither v1_logs
nor opentelemetry.proto.collector.log.v1.LogService_Export
works. I had to use '*'
to get anything matching. API sends OTel via gRPc)
For the record: It's not currently possible, but was considered an oversight by FluentBit project: https://github.com/fluent/fluent-bit/issues/8910
At the time of this writing it was planned for Fluent Bit v3.1.0 milestone (see GitHub ticket).