Search code examples
memorymapreducecpu-usagehadoop-yarn

Get the memory, CPU and disk usage for Yarn application

I want to ask after I've run a Yarn application, how can I get the total memory and CPU usage of that application.

I used to use the resource manager UI to get all the information. But aside from getting this information from the UI, are there any commands that I can use to achieve the information.

Solution

  • Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.

    For e.g. when I type yarn application -status application_1452267331813_0009 (for one of my completed applications), one of the rows returned is:

    Aggregate Resource Allocation : 46641 MB-seconds, 37 vcore-seconds

    This gives an aggregate memory and CPU allocations in seconds. You can check this answer: Aggregate Resource Allocation for a job in YARN, to understand the meaning of this output.

    Apart from this, as of now, there are no other memory or CPU related metrics exposed through CLI.