Search code examples
hadoop-yarnhadoop2

Commands to get the metrics of YARN container's actual memory usage

I have found that a containers actual memory usage stats is being supported by yarn as per this jira ticket ( https://issues.apache.org/jira/browse/YARN-2984). I'd like to know if given an application id can we get the list of memory usages of the containers assigned to that job using any yarn commands ?

Solution

  • I am using Hadoop 2.7.0.

    CLI:

    The YARN CLI does not expose the memory metrics related to a container.

    You can query the status of a container using : yarn container -status <Container ID> command.

    For e.g. when I executed this command, I got the following output:

    CMD PROMPT>yarn container -status container_e55_1451819646302_0001_01_000007

Container Report :
        Container-Id : container_e55_1451819646302_0001_01_000007
        Start-Time : 1451909358911
        Finish-Time : 0
        State : RUNNING
        LOG-URL : http://mballur.myorg.com:50060/node/containerlogs/container_e55_1451819646302_0001_01_000007/mballur
        Host : mballur.myorg.com:45454
        NodeHttpAddress : http://mballur.myorg.com:50060
        Diagnostics : null

    The container report contains (This does not contain any memory related information):

    • Container-Id: ID of the container
    • Start-Time: Start time for the container
    • Finish-Time: Finish time for the conataine. Zero, if the container is still running
    • State: State of the container
    • LOG-URL: Log URL of the container
    • Host: The node ID, where the container is running
    • NodeHttpAddress: HTTP address of the container
    • Diagnostics: Any diagnostics information.

    UI:

    But, in the UI, when you open the page for a particular container, you will see the following details:

    Container Details

    The UI contains Resource: information for the container. There you can see the memory and CPU usage (1228 memory, 1 VCores)