Search code examples
bigdataapache-zookeeperhadoop2

Which hadoop components are installed by default?

I am trying to install Hadoop for development. I did not install Hadoop from Cloudera as I have limited RAM of 8gb and it is not advisable to install CDH VM. Hence I am installing hadoop manually by downloading the tar file and then unpacking it (hadoop-2.8.0.tar.gz. I want to try out the entire ecosystem in Hadoop as I am following Hadoop A definitive guide book.

MY QUESTIONS:

  1. Which ecosystem components are installed by default in such an installation ?

  2. Do I need to install ZooKeeper separately or it is already present?

  3. Would the components clash due to such an installation and what are the precautions that I need to take in order to avoid this?

Solution

  • That archive includes just HDFS and MapReduce framework. It is enough to get you started with hadoop, but it doesn't include any other projects from hadoop ecosystem. You might get away with running Cloudera QuickStart VM on a 8GB virtual machine. If you want to give a try to other projects (e.g. Spark, Hive, HBase...), that's the way to go, as you'll have a hard time integrating all those projects from hadoop ecosystem together by yourself.

    Regarding your other questions:

    • No, zookeeper is not present in the archive that you have downloaded (and you don't need it if you just want to use HDFS and MapReduce on one VM).

    • No, the components won't "clash" (keep in mind, there are just two
      components - HDFS and MapReduce - that you have in your hadoop-2.8
      archive).