Search code examples
hadoophdfshadoop-yarnhadoop2webhdfs

Know the disk space of data nodes in hadoop?


Is there a way or any command using which I can come to know the disk space of each datanode or the total cluster disk space?

I tried the command

dfs -du -h /

but it seems that I do not have permission to execute it for many directories and hence cannot get the actual disk space.


Solution

  • From UI:

    http://namenode:50070/dfshealth.html#tab-datanode

    ---> which will give you all the details about datanode.

    From command line:

    To get disk space of each datanode:

    sudo -u hdfs hdfs dfsadmin -report

    ---> which will give you the details of entire HDFS and the individual datanodes OR

    sudo -u hdfs hdfs dfs -du -h /

    ---> which will give you the total disk usage of each folder under root / directory