Docker: Can't scrape SonarQube with VictoriaMetrics "vmagent", connection refused

Question

I'm in the process of building a docker-compose.yml and want to use VictoriaMetrics own scraper (vmagent) to scrape metrics which SonarQube exposes via a Plugin at /api/prometheus/metrics.

If docker is running, I can access localhost:9000/api/prometheus/metrics and see all my metrics. However, vmagent is throwing the following error every 60 seconds (which means at least my prometheus.yml config is being used correctly):

vmagent            | 2021-02-03T12:02:51.617Z error   VictoriaMetrics/lib/promscrape/scrapework.go:235
                     error when scraping "http://localhost:9000/api/prometheus/metrics" from job "sonarqube" with labels {instance="localhost:9000",job="sonarqube",monitor="codelab-monitor"}:
                     error when scraping "http://localhost:9000/api/prometheus/metrics": dial tcp4 127.0.0.1:9000: connect: connection refused; try -enableTCP6 command-line flag if you scrape ipv6 addresses

I'm not trying to scrape IPv6.

My docker-compose.yml looks like this:

services:

  sonarqube:
    image: sonarqube:8.2-community
    depends_on:
      - postgresdb
    ports:
      - "9000:9000"
    networks:
      - sonarnet
    environment:
      [...]
    volumes:
      [...]

  postgresdb:
    [...]

  victoriametrics:
    container_name: victoriametrics
    image: victoriametrics/victoria-metrics
    ports:
      - 8428:8428
      - 8089:8089
      - 8089:8089/udp
      - 2003:2003
      - 2003:2003/udp
      - 4242:4242
    volumes:
      [...]
    command:
      - '--storageDataPath=/storage'
      - '--graphiteListenAddr=:2003'
      - '--opentsdbListenAddr=:4242'
      - '--httpListenAddr=:8428'
      - '--influxListenAddr=:8089'
    networks:
      - sonarnet
    restart: always

  vmagent:
    container_name: vmagent
    image: victoriametrics/vmagent
    depends_on:
      - "victoriametrics"
    ports:
      - 8429:8429
    volumes:
      - vmagentdata:/vmagentdata
      - ./prometheus:/etc/prometheus
    command:
      - '--promscrape.config=/etc/prometheus/prometheus.yml'
      - '--remoteWrite.url=http://victoriametrics:8428/api/v1/write'
    networks:
      - sonarnet
    restart: always

  vmalert:
    [...]

  alertmanager:
    [...]

  grafana:
    [...]

networks:
  sonarnet:
    driver: bridge

volumes:
  [...]

And my prometheus.yml (which is being read by vmagent correctly I assume, based on the scrape interval being non-default 60 seconds):

global:
  scrape_interval:     60s

  external_labels:
    monitor: 'codelab-monitor'

scrape_configs:
  - job_name: 'sonarqube'
    metrics_path: '/api/prometheus/metrics'
    static_configs:
      - targets: ['localhost:9000']

I suspect my docker networks configuration to be at fault, but I don't have a clue why vmagent can't access http://localhost:9000/api/prometheus/metrics.

Answer 1

Within vmagent container localhost will mean vmagent itself. Try to refer with service name sonarqube:9000 instead of localhost:9000.