Search code examples
dockerapache-sparkopen-telemetryopen-telemetry-collector

Opentelemetry Collector - Connection refused problem

I recently started using Opentelemetry, so i have some problems about configuring a Collector. The situation is :

  • I have a Docker container where an Apache Spark cluster is running. The Spark master is reachable from browser with the link http://127.0.0.1:9090/
  • I have a Docker container where an Opentelemetry Collector is running. This container is created from the official image otel/opentelemetry-collector-contrib:0.98.0

They running on the same network, i verified this by using docker network inspect. I'm trying to configure the Opentelemetry Collector to monitoring the Apache Spark cluster and the applications running on it through the collection of performance metrics but i have a problem. I'm using this configuration for the collector:

receivers:
  apachespark:
    collection_interval: 10s
    endpoint: http://localhost:9090/
    application_names:  

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    metrics:
      receivers: [apachespark]
      exporters: [debug]

And I obtain the following error :

2024-04-23T15:26:34.466Zerrorscraperhelper/scrapercontroller.go:197Error scraping metrics{"kind": "receiver", "name": "apachespark", "data_type": "metrics", "error": "failed to retrieve app ids\nGet "http://localhost:9090//api/v1/applications": dial tcp 127.0.0.1:9090: connect: connection refused", "scraper": "apachespark"} go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:197 go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1 go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:173

How can I solve ? Thanks a lot for supporting me and I apologize in advance if I made a beginner's mistake, but it is a new topic for me.

Solution

  • Try giving your PCs local IP instead of localhost in this:

    receivers:
  apachespark:
    collection_interval: 10s
    endpoint: http://**<your PCs Local IP address here>**:9090/
    application_names: