How can I monitor a ASP.Net Core app with dockerized Prometheus (and Grafana)?

Question

I would like to have Prometheus and Grafana running on my developer machine using docker-images / docker-for-windows.

I have system-under-development, ASP.Net core, running on localhost:5001 and metrics are showing just fine on https://localhost:5001/metrics.

Docker-compose.yml and prometheus.yml listed below.

• If I include network_mode: host in docker-compose.yml, I can't access Prometheus on my physical machine via localhost:9090
• If I exclude network_mode and instead use ports: , I can access Prometheus on my physical machine via localhost:9090, but checking http://localhost:9090/targets, it shows https://localhost:5001/metrics as being down.

What am I doing wrong? Any comments welcome!

docker-compose.yml:

version: '3.8'
services:
  prometheus:
    image: prom/prometheus
    container_name: gradle_docker-prometheus
    #network_mode: host
    ports:
      - 9090:9090
    volumes:
      - prometheus-storage:/var/lib/prometheus
      - /c/Data/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
  grafana:
    image: grafana/grafana
    container_name: gradle_docker-grafana
    ports:
      - "3000:3000"
    volumes:
      - grafana-storage:/opt/grafana/data
    depends_on:
      - prometheus

volumes:
  prometheus-storage: {}
  grafana-storage: {}

prometheus.yml:

global:
  scrape_interval:     15s
  evaluation_interval: 15s

  external_labels:
      monitor: 'my-project'

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 10s
    scheme: http
    static_configs:
         - targets: ['localhost:9090','cadvisor:8080','node-exporter:9100', 'nginx-exporter:9113']
  - job_name: '.Net'
    scrape_interval: 10s
    scheme: https
    static_configs:
         - targets: ['localhost:5001']

Answer 1

Do not use host network mode on Windows, it is only supported on Linux. What you need is to change the target address:

  - job_name: '.Net'
    scrape_interval: 10s
    scheme: https # You may have to change this to 'http'
                  # or you'd have to create a certificate 
                  # with `host.docker.internal`
    static_configs:
         - targets: ['host.docker.internal:5001']

host.docker.internal is a special address to connect to the Docker Host, since localhost inside a container is just the container itself.

To summarize what goes below in comments: after changing the target to host.docker.internal, ensure that your application allows to connect with that host. Run

curl http://localhost:5001/ -H "Host: host.docker.internal"

and check the answer. If you have an error containing something like this:

The request hostname is invalid.

Then you've got to find out where the host filter is (it is probably an array containing localhost in it) and add the new host (host.docker.internal) there.