Search code examples
open-telemetryopen-telemetry-collectoropen-telemetry-java

How to Transform OpenTelemetry Collector Logs into JSON Format?


i'm currently working with the OpenTelemetry Collector and would like to format its logs in JSON. I've been using a YAML configuration file for the collector, but I'm unsure how to specify that the logs should be outputted in JSON format.

Here's a simplified version of my current configuration:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:6666

exporters:
  logging:
    loglevel: warn
  otlphttp:
    endpoint: ${OTEL_EXPORTER_OTLP_ENDPOINT}
    auth:
      authenticator: basicauth/client

extensions:
  basicauth/client:
    client_auth: 
      username: ${USERNAME}
      password: ${PASSWORD}

service:
  extensions: [basicauth/client]
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [otlphttp]
    traces:
      receivers: [otlp]
      exporters: [otlphttp, logging]

Solution

  • To configure the OpenTelemetry Collector to output its own logs in JSON format, use the following configuration:

    service:
      telemetry:
        logs:
          encoding: json
    

    For more information, see the docs at https://opentelemetry.io/docs/collector/configuration/#telemetry.

    Here's the configuration I used (a shorter version of the config in the question) and the output I got from running it. As you can see, the logs are formatted in JSON.

    receivers:
      otlp:
        protocols:
          http:
            endpoint: 0.0.0.0:6666
    exporters:
      logging:
        loglevel: warn
    service:
      pipelines:
        metrics:
          receivers: [otlp]
          exporters: [logging]
        traces:
          receivers: [otlp]
          exporters: [logging]
      telemetry:
        logs:
          encoding: json
    
    C:\>otelcol-contrib_0.96.0_windows_amd64.exe --config .\config.yaml
    {"level":"info","ts":1710634949.465228,"caller":"[email protected]/telemetry.go:55","msg":"Setting up own telemetry..."}
    {"level":"info","ts":1710634949.4659326,"caller":"[email protected]/telemetry.go:97","msg":"Serving metrics","address":":8888","level":"Basic"}
    {"level":"info","ts":1710634949.4665034,"caller":"[email protected]/exporter.go:275","msg":"Deprecated component. Will be removed in future releases.","kind":"exporter","data_type":"traces","name":"logging"}
    {"level":"warn","ts":1710634949.4676774,"caller":"common/factory.go:68","msg":"'loglevel' option is deprecated in favor of 'verbosity'. Set 'verbosity' to equivalent value to preserve behavior.","kind":"exporter","data_type":"traces","name":"logging","loglevel":"warn","equivalent verbosity level":"Basic"}
    {"level":"info","ts":1710634949.469333,"caller":"[email protected]/exporter.go:275","msg":"Deprecated component. Will be removed in future releases.","kind":"exporter","data_type":"metrics","name":"logging"}
    {"level":"info","ts":1710634949.4708097,"caller":"[email protected]/service.go:143","msg":"Starting otelcol-contrib...","Version":"0.96.0","NumCPU":8}
    {"level":"info","ts":1710634949.4715457,"caller":"extensions/extensions.go:34","msg":"Starting extensions..."}
    {"level":"warn","ts":1710634949.472695,"caller":"[email protected]/warning.go:42","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. Enable the feature gate to change the default and remove this warning.","kind":"receiver","name":"otlp","data_type":"traces","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks","feature gate ID":"component.UseLocalHostAsDefaultHost"}
    {"level":"info","ts":1710634949.474763,"caller":"[email protected]/otlp.go:152","msg":"Starting HTTP server","kind":"receiver","name":"otlp","data_type":"traces","endpoint":"0.0.0.0:6666"}
    {"level":"info","ts":1710634949.4759579,"caller":"[email protected]/service.go:169","msg":"Everything is ready. Begin running and processing data."}
    {"level":"warn","ts":1710634949.4768088,"caller":"localhostgate/featuregate.go:63","msg":"The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default.","feature gate ID":"component.UseLocalHostAsDefaultHost"}