Search code examples
bashprometheusfederation

Prometheus share slaves is set as follows, the status is up, but no data can be searched. Where does my error occur?


I want to build horizontal extensions, implement a total prometheus, and monitor two child prometheus nodes. For example: Nodes A, B: Monitor Node_exporter and mysql_export data;

The total Prometheus is on the host computer C, and the child nodes are on Hosts A and B respectively.

The master node is configured as follows:

prometheus.yml(host_C):

global:

rule_files:
#  - node_rules/zep.test.rules 


scrape_configs:
  - job_name: slaves

    honor_labels: true
    scrape_interval: 1s
    metrics_path: /federate

    params:
      match[]:
         - '{__name__=~"^job:.*"}'
         - '{__job__=~"^job:.*"}'

    static_configs:
      - targets:
         - hostA_ip:9090
         - hostB_ip:9090

The child nodes are configured as follows:

slaves1.yml(host_A):

global:

  external_labels:
    slave: 0 


rule_files:

scrape_configs:
  - job_name: myjob_1
    scrape_interval: 1s

    file_sd_configs:
      - files: ['./mytest.json']

    relabel_configs:
      - source_labels: [__address__]
        modulus:       2   
        target_label:  __tmp_hash
        action:        hashmod

      - source_labels: [__tmp_hash]
        regex:         ^0$ 
        action:        keep

slaves2.yml(host_B):

global:

  external_labels:
    slave: 1 


rule_files:

scrape_configs:
  - job_name: myjob_2
    scrape_interval: 1s

    file_sd_configs:
      - files: ['./mytest.json']

    relabel_configs:
      - source_labels: [__address__]
        modulus:       2   
        target_label:  __tmp_hash
        action:        hashmod

      - source_labels: [__tmp_hash]
        regex:         ^1$ 
        action:        keep

mytest.json:

[{
    "targets": [
        "hostA_ip:9100",
        "hostA_ip:9104"
    ],
    "labels": {
        "services": "dba_test"
    }
}]

run it :

./prometheus --web.listen-address="hostA_ip:9090" --storage.tsdb.path="global_data/" --config.file="prometheus.yml" --web.enable-admin-api
./prometheus --web.listen-address="hostB_ip:9090" --storage.tsdb.path="data1/" --config.file="slave1.yml" --web.enable-admin-api
./prometheus --web.listen-address="hostC_ip:9090" --storage.tsdb.path="data2/" --config.file="slave2.yml" --web.enable-admin-api

Solution

  • The reason for this problem is because wildcards do not match. The official website provides ___job____, but the actual use is job.

    Specific will also look at the 9090 under the status of the Targets specific configuration page, not all in accordance with the official to provide configuration