Search code examples
spring-bootopen-telemetrygrafana-lokiotlp-grpc

Unable to map otlp resource attributes into loki labels


I'm using a spring boot application which sends telemetry data including logs to opentelemetry collector using opentelemetry agent.

The logs are successfully transferred to opentelemetry collector and received at the grafana loki but the Loki labels are not updated with OTLP resource and log attributes.

current config -> Loki Label -> host_name: host.name (as mentioned in otlp resource label) expected : Loki Label -> host_name: 06605506424f or any other attribute name eg : service.name

opentelemetry collector config file :

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:  
  attributes:
    actions:
    - action: insert
      key: loki.attribute.labels
      value: host.name
  
  resource:
    attributes:
    - action: insert
      key: loki.attribute.labels
      value: service.name
    - action: insert
      key: loki.resource.labels
      value: host.name, container.id  
      
  batch:

exporters:
  jaeger:
    endpoint: "jaeger:14250"
    tls:
      insecure: true
  logging:
    loglevel: debug    
   
  loki:
    endpoint: "http://loki:3100/loki/api/v1/push"
    
  prometheus:
    endpoint: "0.0.0.0:8889"
    namespace: "default"    

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [health_check,pprof,zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]
    logs:
      receivers: [otlp]
      processors: [resource,attributes,batch]
      exporters: [logging,loki] 

opentelemetry collector logs :

Resource SchemaURL: https://opentelemetry.io/schemas/1.12.0
Resource attributes:
     -> container.id: STRING(06605506424f3396e1c92aeedc22337288177bc25c97af21beeda42a898e9be4)
     -> host.arch: STRING(amd64)
     -> host.name: STRING(06605506424f)
     -> os.description: STRING(Linux 5.10.16.3-microsoft-standard-WSL2)
     -> os.type: STRING(linux)
     -> process.command_line: STRING(/usr/lib/jvm/java-1.8-openjdk/jre:bin:java -javaagent:/dir/opentelagent.jar)
     -> process.executable.path: STRING(/usr/lib/jvm/java-1.8-openjdk/jre:bin:java)
     -> process.pid: INT(1)
     -> process.runtime.description: STRING(IcedTea OpenJDK 64-Bit Server VM 25.212-b04)
     -> process.runtime.name: STRING(OpenJDK Runtime Environment)
     -> process.runtime.version: STRING(1.8.0_212-b04)
     -> service.name: STRING(spring-api-gateway)
     -> telemetry.auto.version: STRING(1.18.0)
     -> telemetry.sdk.language: STRING(java)
     -> telemetry.sdk.name: STRING(opentelemetry)
     -> telemetry.sdk.version: STRING(1.18.0)
     -> loki.attribute.labels: STRING(service.name)
     -> loki.resource.labels: STRING(host.name, container.id)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope com.apigateway.SpringApiGatewayApplication 
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2022-10-04 10:02:10.386 +0000 UTC
SeverityText: INFO
SeverityNumber: SEVERITY_NUMBER_INFO(9)
Body: Started SpringApiGatewayApplication in 10.038 seconds (JVM running for 16.182)
Attributes:
     -> loki.attribute.labels: STRING(host.name)
Trace ID: 
Span ID: 
Flags: 0
        {"kind": "exporter", "data_type": "logs", "name": "logging"}

grafana Loki

enter image description here:


Solution

  • I think this is a known bug in the Loki exporter for OpenTelemetry. The workaround is to define the resource labels in the logRecord attribute section until the issue is fixed