Search code examples
ganglia

clustering gmetad and pull data from other gmetad

I am new to ganglia and trying to play around with clustering of gmetad for my project

I have a cluster of around 500 nodes and hence i want to distribute the load of gmetad(aggregator) to multiple nodes and then using a central gmetad /gweb to show all the results.

Also, i have a limitation that I can not have all the data from all the nodes on one machine, so I wanted to know if there any options like below,

  1. If I configure cluster of gmetad, will my central gmetad contain data in rrd(central) from all the other gmetads?
  2. is it possible to stream the data real-time on to the central gmetad for gweb to display instead of storing it in rrd?

Please find the sample configuration below,

central configuration

data_source "grid1" xx.xx.xxx.xxx
gridname "central"
trusted_hosts xx.xx.xxx.xxx
all_trusted on
rrd_rootdir "/ganglia/rrds"
case_sensitive_hostnames 1

grid1 configuration

data_source xx.xx.xxx.xxx
gridname "grid1"
rrd_rootdir "/ganglia/rrds"
case_sensitive_hostnames 1

I have tried with above configuration and able to see the data on web but data from all the gmetads are copied to central gmetad.

Thanks in advance

Solution

    1. Yes. When you configure an entry of data_source in gmetad.conf, A TCP polling thread starts to fetch metrics data from gmond or gmetad, then store the polled metrics data in rrd files host by host, no matter it polls gmond or gmetad.

    2. No. gweb application depends on the rrd files which is the single source of metrics data. However, you can avoid using rrd files by integrating with graphite, gmetad will send the polled data directly to the configured carbon server. see my answer to the question Sending Data from ganglia to graphite

    One solution is replacing those sub-gmetad with gmond in mute mode which means this kind of gmond is used to collect metrics from other gmond in same cluster, then let your single gmetad pull these mute gmond, so that there's only one gmetad to pull all metrics into rrds in all clusters.

    • you need to configure mute gmond via mute = yes
    • only configure these mute gmond in data_source entries of your single gmetad