Search code examples
hadoopcluster-computingbiginsights

Hadoop Namenode without HDFS storage

I have installed a hadoop cluster with total 3 machines, with 2 nodes acting as datanodes and 1 node acting as Namenode and as well as a Datanode. I wanted to clear certain doubts regarding hadoop cluster installation and architecture. Here is a list of questions I am looking answers for----

  1. I uploaded a data file around 500mb size in the cluster and then checked the hdfs report. I noticed that the namenode I made is also occupying 500mb size in the hdfs, along with datanodes with a replication factor of 2. The problem here is that I want the namenode not to store any data on it, in short i dont want it to work as a datanode as it is also storing the file I am uploading. So what is the way of making it only act as a Master Node and not like a datanode?

I tried running the command hadoop -daemon.sh stop on the Namenode to stop the datanode services on it but it wasnt of any help.

  1. How much metadata does a Namenode generate for a filesize typically of 1 GB? Any approximations?
Solution

  • Go to conf directory inside your $HADOOP_HOME directory on your master. Edit the file named slaves and remove the entry corresponding to your name node from it. This way you are only asking the other two nodes to act as slaves and name node as only the master.