I'm using a spring boot application which sends telemetry data including logs to opentelemetry collector using opentelemetry agent.
The logs are successfully transferred to opentelemetry collector and received at the grafana loki but the Loki labels are not updated with OTLP resource and log attributes.
current config -> Loki Label -> host_name: host.name (as mentioned in otlp resource label) expected : Loki Label -> host_name: 06605506424f or any other attribute name eg : service.name
opentelemetry collector config file :
receivers:
otlp:
protocols:
grpc:
http:
processors:
attributes:
actions:
- action: insert
key: loki.attribute.labels
value: host.name
resource:
attributes:
- action: insert
key: loki.attribute.labels
value: service.name
- action: insert
key: loki.resource.labels
value: host.name, container.id
batch:
exporters:
jaeger:
endpoint: "jaeger:14250"
tls:
insecure: true
logging:
loglevel: debug
loki:
endpoint: "http://loki:3100/loki/api/v1/push"
prometheus:
endpoint: "0.0.0.0:8889"
namespace: "default"
extensions:
health_check:
pprof:
zpages:
service:
extensions: [health_check,pprof,zpages]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [jaeger]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheus]
logs:
receivers: [otlp]
processors: [resource,attributes,batch]
exporters: [logging,loki]
opentelemetry collector logs :
Resource SchemaURL: https://opentelemetry.io/schemas/1.12.0
Resource attributes:
-> container.id: STRING(06605506424f3396e1c92aeedc22337288177bc25c97af21beeda42a898e9be4)
-> host.arch: STRING(amd64)
-> host.name: STRING(06605506424f)
-> os.description: STRING(Linux 5.10.16.3-microsoft-standard-WSL2)
-> os.type: STRING(linux)
-> process.command_line: STRING(/usr/lib/jvm/java-1.8-openjdk/jre:bin:java -javaagent:/dir/opentelagent.jar)
-> process.executable.path: STRING(/usr/lib/jvm/java-1.8-openjdk/jre:bin:java)
-> process.pid: INT(1)
-> process.runtime.description: STRING(IcedTea OpenJDK 64-Bit Server VM 25.212-b04)
-> process.runtime.name: STRING(OpenJDK Runtime Environment)
-> process.runtime.version: STRING(1.8.0_212-b04)
-> service.name: STRING(spring-api-gateway)
-> telemetry.auto.version: STRING(1.18.0)
-> telemetry.sdk.language: STRING(java)
-> telemetry.sdk.name: STRING(opentelemetry)
-> telemetry.sdk.version: STRING(1.18.0)
-> loki.attribute.labels: STRING(service.name)
-> loki.resource.labels: STRING(host.name, container.id)
ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope com.apigateway.SpringApiGatewayApplication
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2022-10-04 10:02:10.386 +0000 UTC
SeverityText: INFO
SeverityNumber: SEVERITY_NUMBER_INFO(9)
Body: Started SpringApiGatewayApplication in 10.038 seconds (JVM running for 16.182)
Attributes:
-> loki.attribute.labels: STRING(host.name)
Trace ID:
Span ID:
Flags: 0
{"kind": "exporter", "data_type": "logs", "name": "logging"}
grafana Loki
I think this is a known bug in the Loki exporter for OpenTelemetry. The workaround is to define the resource labels in the logRecord
attribute section until the issue is fixed