Jaeger agent not receiving spans from Node.JS client

I am working on a backend composed of multiple microservices, and I want to be able to view the spans in the Jaeger UI. I use docker-compose to run my containers, including jaeger, and opentelemetry to generate and send spans. I have followed the troubleshooting guide up to and including the logging reporter.

This is my first time working with jaeger and this kind of architecture so I feel a bit lost at this point.

Here are some relevant parts of my code, some logs and screenshots :

Docker-compose.yaml

x-aliases :
  - &jaeger_envs
    JAEGER_SAMPLER_TYPE: const
    JAEGER_SAMPLER_PARAM: 1
    JAEGER_ENDPOINT: http://127.0.0.1:14268/api/traces

api-gateway:
    build: ./src/api-gateway/
    container_name: api-gateway
    restart: always
    environment:
      MONGO_DB_URL: user-db:27017
      QUEUE_SERVICE_URL: queue-service:5001
      STATUS_SERVICE_URL: status-service:5002
      TICKET_SERVICE_URL: ticket-service:5003
      JAEGER_REPORTER_LOG_SPANS: 'true'
      <<: *api_gateway_envs
      <<: *jaeger_envs
    ports:
      - "127.0.0.1:5000:5000"
    depends_on:
      - user-db
      - queue-service
      - status-service
      - ticket-service

jaeger:
    image: jaegertracing/all-in-one:latest
    container_name: jaeger
    restart: always
    environment:
      COLLECTOR_ZIPKIN_HOST_PORT: 9411
    ports:
      - "127.0.0.1:5775:5775/udp"
      - "127.0.0.1:6831:6831/udp"
      - "127.0.0.1:6832:6832/udp"
      - "127.0.0.1:5778:5778"
      - "127.0.0.1:16686:16686"
      - "127.0.0.1:14268:14268"
      - "127.0.0.1:14269:14269"
      - "127.0.0.1:14250:14250"
      - "127.0.0.1:9411:9411"

/src/ticket_service/tracing.ts

import { NodeSDK } from "@opentelemetry/sdk-node";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { JaegerExporter } from '@opentelemetry/exporter-jaeger';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { Resource } from '@opentelemetry/resources';

const jaegerExporter = new JaegerExporter({
  endpoint: 'http://127.0.0.1:14268/api/traces', //hardcoded to make sure this is not an env var issue
});

const resource = new Resource({
  [SemanticResourceAttributes.SERVICE_NAME]: 'api-gateway',
});

const sdk = new NodeSDK({
  traceExporter: jaegerExporter,
  instrumentations: [getNodeAutoInstrumentations()],
  resource: resource
});

sdk.start();
console.log("tracer initialized");

I run the api-gateway micro-service with cross-env NODE_ENV=production NODE_PATH=dist/ node -r ./dist/src/tracing.js ./dist/src

Api-gateway logs :

{"body":{"email":"test@test.com","password":"***"},"duration":30.636999999999997,"level":"debug","message":"POST /api/auth/sign-in - 401 - 30.64 ms","method":"POST","originalUrl":"/api/auth/sign-in","params":{},"path":"/api/auth/sign-in","remoteIp":"::ffff:172.18.0.1","remotePort":59534,"span_id":"596fa50386bd55a1","state":"outgoing","status":401,"timestamp":"2022-03-23T10:34:22.102Z","trace_flags":"01","trace_id":"8278acbbfb7c59ffb2296d96660f005e"}

{"body":{"email":"test@test.com","password":"***"},"duration":5.9342999999999995,"level":"debug","message":"POST /api/auth/sign-in - 401 - 5.93 ms","method":"POST","originalUrl":"/api/auth/sign-in","params":{},"path":"/api/auth/sign-in","remoteIp":"::ffff:172.18.0.1","remotePort":59534,"span_id":"704d7a23ae3cb1e1","state":"outgoing","status":401,"timestamp":"2022-03-23T10:34:23.562Z","trace_flags":"01","trace_id":"7dd1c34706602bab47e5ca78a78dd7cc"}

{"body":{"email":"test@test.com","password":"***"},"duration":3.6582999999999997,"level":"debug","message":"POST /api/auth/sign-in - 401 - 3.66 ms","method":"POST","originalUrl":"/api/auth/sign-in","params":{},"path":"/api/auth/sign-in","remoteIp":"::ffff:172.18.0.1","remotePort":59534,"span_id":"eace80cc6146a8fb","state":"outgoing","status":401,"timestamp":"2022-03-23T10:34:24.750Z","trace_flags":"01","trace_id":"15d1e13cd8e17558bd6624e069336219"}

Jaeger Agent container log related to the HTTP endpoint

{"level":"info","ts":1648028283.2791674,"caller":"flags/admin.go:115","msg":"Starting admin HTTP server","http-addr":":14269"}
{"level":"info","ts":1648028283.2952828,"caller":"server/http.go:48","msg":"Starting jaeger-collector HTTP server","http host-port":":14268"}

Screenshot of the Jaeger UI

Jaeger UI

Any help or idea would be greatly appreciated ! Have a good day !

Solution

You have to think of a container as an individual minimal host. And in that case, when you say to your ticket_service app to call localhost it will call itself, which is not what we want.

Whenever using docker-compose, a docker network is created and the containers are configured to use it.
You can use that network to make your containers communicate with each other by their names.

In your case, as the Jaeger container is called jaeger, you have to configure the endpoint of your JaegerExporter as follows:

const jaegerExporter = new JaegerExporter({
  endpoint: 'http://jaeger:14268/api/traces'
});