Prometheus: How to get Consul nodes from multiple datacenters

Question

I'd like to use Prometheus's Consul integration to auto-discover all my consul nodes. At the moment, my Prometheus server only receives nodes from a single datacenter in Consul eventhough I never actually specified which datacenter to use in the configuration (I guess it just picked the one that my consul-client (installed on my prometheus server) is part of).

How do I get all the nodes from all the datacenters that consul is aware of?

Answer 1

The solution we found was to use the 'datacenter' keyword in the configuration - and to explicitly list of the datacenters we have. Its not optimal (because we might add/remove datacenters in the future) but it does work. below is an example:

scrape_configs:
  - job_name: 'consul'
    consul_sd_configs:
       - server:   '0.0.0.0:8500'
         datacenter: 'datacenter-name-1'
       - server:   '0.0.0.0:8500'
         datacenter: 'datacenter-name-2'
       - server:   '0.0.0.0:8500'
         datacenter: 'datacenter-name-3'