Search code examples
javamavenhadoophadoop-yarnhadoop2

job history webui-19888 in hadoop2.7.2 multinode shows noting after job has finished

after I ran a job in hadoop2.7.2 with 2 slaves ,I can see a result in terminal, but in master:19888/jobhistory is nothing.I uploaded the picture and my config files. job history webui 19888

and here is my config files. core.site.xml

<property>
    <name>fs.default.name</name>
    <value>hdfs://master:9000</value>    
</property>

   <property>
    <name>hadoop.tmp.dir</name>
    <value>/app/hadoop/tmp</value> 
</property>

hdfs.site.xml

<property>
<name>dfs.replication</name>
<value>2</value>
</property>

<property>
<name>dfs.name.dir</name>
<value>file:/usr/local/hadoop/hadoop_tmp/hdfs/namenode</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>file:/usr/local/hadoop/hadoop_tmp/hdfs/datanode</value>
</property>

mapred.site.xml

<property>
    <name>mapred.job.tracker</name>
    <value>master:54311</value>    
</property>

<property>
    <name>mapred.framework.name</name>
    <value>yarn</value>
</property>

<property>
<name>mapred.jobhistory.address</name>
<value>master:10020</value>
</property>

yarn.site.xml

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>


<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8025</value>
</property>

<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8035</value>
</property>

<property>
<name>yarn.resourcemanager.address</name>
<value>master:8050</value>
</property>
Solution

  • The property is mapreduce.framework.name in mapred-site.xml

    <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
</property>

    The current property mapred.framework.name is not valid and is not recognized. Thus, the jobs are being submitted in local mode. Locally run jobs are not tracked by jobhistoryserver.

    Modify the property name and restart the services.