Search code examples
hadoophdfscapacity

HDFS capacity mismatch

I'm doing some tests on Hadoop and the HDFS is almost full after those tests. So I deleted a lot of test files recursively but the released space is not as much as I expected.

Running hadoop fs -du -s / command, it reports 876447242289. Means about 0.8 TB. But in Name Node UI, it reports 12 TB still in used.

Configured Capacity  :   19.89 TB
DFS Used         :   12.15 TB
Non DFS Used     :   801.66 GB
DFS Remaining    :   6.95 TB
DFS Used%        :   61.11 %
DFS Remaining%   :   34.96 %
Block Pool Used  :   12.15 TB
Block Pool Used% :   61.11 %

I Ran `df -h. in each data node and I found the same result as Name Node UI.

I checked the .Trash folder and nothing is there. Looks like many files are removed from Name Node, but still not deleted in Data Node. Can anyone help on this?

Solution

  • I found that the space will be released gradually. After 8 hours, all the spaces that were held by deleted files are released.