Search code examples
hadoopmapreducehadoop-yarn

What does YARN_HEAPSIZE mean in yarn-env.sh?

What does setting "YARN_HEAPSIZE=500" in yarn-env.sh mean?

What does it mean to set the heap size of HADOOP's main daemon? For example , (yarn-env.sh) YARN_HEAPSIZE=500 YARN_RESOURCEMANAGER_OPTS=-Xmx4g YARN_NODEMANAGER_OPTS=-Xmx1g

(hadoop-env.sh) HADOOP_NAMENODE_OPTS=-Xmx4g HADOOP_DATANODE_OPTS=-Xmx4g

(mapred-env.sh) HADOOP_JOB_HISTORYSERVER_HEAPSIZE=250

How are the above settings different from yarn.scheduler.minimum-allocation-mb,yarn.scheduler.maximum-allocation-mb,yarn.nodemanager.resource.memory-mb in yarn-site.xml ?

Solution

  • They each set the JVM daemon heap sizes.

    The others you've mentioned in the XML files are for applications that get launched within YARN.

    The explanation for each is already in the Hadoop website under the cluster installation pages.