Search code examples
javaeclipsehadoophadoop2

how turn off speculative execution in hadoop source code in eclipse

I need to turn it off in hadoop source code ,and I can not find its package in eclipse. https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/Job.html#setReduceSpeculativeExecution%28boolean%29

Solution

  • If you want to disable Speculative Execution, add these properties in mapred-site.xml.

    <property>
   <name>mapreduce.map.speculative</name>
   <value>false</value>
</property>
<property>
   <name>mapreduce.reduce.speculative</name>
   <value>false</value>
</property>

    Or at the Job level, set the properties in the Driver class.

    Configuration conf = new Configuration();
conf.set(“mapreduce.map.speculative”,false);
conf.set(“mapreduce.reduce.speculative”,false);

    For its source code refer these: hadoop-mapreduce-project, Job and JobConf