Search code examples
apache-sparkmesos

How to get Mesos Agents Framework Executor Memory

Inside Mesos Web UI I can see memory usage of my Spark executors in a table

Agents -> Framework -> Executors

There is a table listing all executors for my Spark driver and their memory usage is indicated in column Mem (Used / Allocated).

Is there a way to obtain this number directly via a link and if yes how?

For example I can obtain a bunch of Mesos metrics via http://IP/mesos/metrics/snapshot but memory usage of executors is not one of them.

Solution

  • Found the answer myself. For each worker/agent on which executors may run, direct access to memory info is here:

    http://IP_of_worker1:5051/slave(1)/monitor/statistics
    http://IP_of_worker2:5051/slave(1)/monitor/statistics
    etc

    The content is in the form of a json and framework_id allows to find the related executors and their memory consumption, cpu usage etc what is given in the table.