As the title states, I have some client applications sending metrics, traces and logs to opentelemtry collector, and the collector should then export mentioned signals to a jaeger instance.
For some reason It is failing with a not very clear error and i'm stuck.
Here is the Docker compose:
version: '3.9'
x-default-logging: &logging
driver: "json-file"
options:
max-size: "5m"
max-file: "2"
services:
# ********************
# Telemetry Components
# ********************
# Jaeger
jaeger:
image: jaegertracing/all-in-one:1.50
container_name: jaeger
deploy:
resources:
limits:
memory: 500M
restart: unless-stopped
ports:
- "16686:16686" # Jaeger UI
- "4317" # OTLP gRPC default port
- "4318" # OTLP HTTP default port (protobuf)
- "14250"
environment:
- COLLECTOR_OTLP_ENABLED=true
logging: *logging
# OpenTelemetry Collector
otelcol:
image: otel/opentelemetry-collector-contrib:0.87
container_name: otel-col
deploy:
resources:
limits:
memory: 525M
restart: unless-stopped
command: [ "--config=/etc/otelcol-config.yml" ]
volumes:
- ./otel/otelcol-config.yml:/etc/otelcol-config.yml
ports:
- "4317:4317" # OTLP over gRPC receiver
- "4318:4318" # OTLP over HTTP receiver
- "9464" # Prometheus exporter
- "8888" # metrics endpoint
depends_on:
- jaeger
logging: *logging
And here is otelcol-config.yml:
receivers:
otlp:
protocols:
grpc:
http:
cors:
allowed_origins:
- "http://*"
- "https://*"
exporters:
debug:
verbosity: detailed
otlp:
endpoint: "jaeger:4317"
tls:
insecure: true
processors:
batch:
service:
telemetry:
logs:
level: debug
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug, otlp]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug, otlp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [debug, otlp]
I can confirm the collector succesfully receives all the signals (it exports to stdout of collector), but when exporting to Jaeger I get the following error:
2023-10-17T01:44:36.404Z error exporterhelper/retry_sender.go:145 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "Permanent error: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.metrics.v1.MetricsService", "dropped_items": 35}
otel-col | go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
otel-col | go.opentelemetry.io/collector/[email protected]/exporterhelper/retry_sender.go:145
otel-col | go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send
otel-col | go.opentelemetry.io/collector/[email protected]/exporterhelper/metrics.go:176
otel-col | go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1
otel-col | go.opentelemetry.io/collector/[email protected]/exporterhelper/queue_sender.go:126
otel-col | go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1
otel-col | go.opentelemetry.io/collector/[email protected]/exporterhelper/internal/bounded_memory_queue.go:52
It seems that the function is not implemented on Jaeger side ? I've looked trough documentation and examples on both Jaeger and OpenTelemetry website / Github and they all do it this way, but I can't get it to work.
Jaeger can only receive traces, not metrics or logs. If you remove otlp from the exporter pipelines in metrics and logs, the error should go away.
If you do want to export metrics and logs, you'll need to install compatible backends such as Prometheus and Loki, and use the appropriate exporters.