Search code examples
javahivehadoop-yarnhadoop2

how to increase java heap size in Hadoop

I am using Hadoop 2.6.0 version and trying to run Hive insert into table where i got the JAVA Heap error.

Is there any way I can increase the heap size in hadoop through out the cluster?

Thanks in advance

Solution

  • For that you may execute the following before executing hadoop command:

    export HADOOP_HEAPSIZE=4096

    Alternatively, you can achieve the same thing by adding the following permanent setting in your mapred-site.xml file, this file lies in HADOOP_HOME/conf/ :

    <property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx4096m</value>
</property>

    Look also here: https://www.mapr.com/blog/how-to-avoid-java-heap-space-errors-understanding-and-managing-task-attempt-memory