Search code examples
mapreducehadoop-yarnbranch-and-bound

Getting started with MapReduce version 2

Good morning,

I did not succeed to find a mapReduce example on YARN (i.e. The 2nd version of MapReduce), the one that is always presented is WordCount which is just the very same code as the one presented on the first version of MapReduce. Even "Hadoop: the definitive guide" doesn't have codes in YARN!

Can you provide me with a code that shows me the difference of writing a mapReduce code in the previous version and the newest version?

In fact, I was trying to write a branch and bound code on MR1 but then I saw that YARN can make the things easiser thanks to BranchReduce.

Any help is appreciated, Thanks in advance

Solution

  • You could compile a program written for MRv1 with Yarn(MRv2) without modifying any single line of the source code. It is completely source-code compatible.

    Here is the Yarn Example: http://wiki.apache.org/hadoop/WordCount

    Here is the Map Red 1 Example: https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Example%3A+WordCount+v1.0

    Some obvious differences to note at the API level:
    - New API uses abstract classes over interfaces
    - The mapreduce package is different