Search code examples
hadoophdfsubuntu-16.04hadoop3

Datanode started but not shown in dfsadmin -report

I am trying to install Hadoop 3.1.0 into two virtual machines:

The first machine contains one name node and one data node, the second contains one data node.

I followed this article Install Hadoop 3.0.0 multi-node cluster on Ubuntu. And every goes fine until i executed the start-dfs.sh and start-yarn.sh commands.

When i run the Jps command on Name node it shows the following results:

16488 NameNode  
16622 DataNode  
17215 NodeManager  
17087 ResourceManager  
17530 Jps  
16829 SecondaryNameNode

And when I run it on the Data node it shows:

2306 DataNode
2479 NodeManager
2581 Jps

But the weird thing is that when executing hdfs dfsadmin -report command it only shows Live datanodes (1) which is the local datanode.

I made a lot of searches; I didn't find anything useful.

I tried to:

  1. Execute stop-all.sh
  2. Delete all files in Namenode and datanode directories
  3. delete all hadoop related files in temp directory
  4. executed hdfs namenode -format command

There is a similar question here: Hadoop UI shows only one Datanode In which they suggested to change the hosts name. In my Namenode /etc/hosts file contains the following lines:

127.0.0.1   localhost
127.0.1.1   hadoop-Namenode
192.168.75.131  hadoop-Namenode
192.168.75.132  hadoop-datanode-1

And in the Datanode /etc/hosts i have:

127.0.0.1   localhost
127.0.1.1   hadoop-datanode-1
192.168.75.131  hadoop-Namenode
192.168.75.132  hadoop-datanode-1

Any suggestions?

Solution

  • The issue was solved by removing 127.0.1.1 lines from both machines

    127.0.1.1   hadoop-Namenode
127.0.1.1   hadoop-datanode-1

    These lines was causing a conflict when trying to resolve the host name.