Search code examples
hadoophdfshortonworks-data-platform

How to free Non DFS Used space with Hortonworks hdp SSH client?


I'm using HDP for self-study to learn Big data basics. Today I've faced the following: HDFS disk usage is 91%. With Non DFS Used 31.2 GB / 41.6 GB (74.96%).

enter image description here What exactly should I do to free disk space? Is it possible to do from sandbox hdp SSH client? I'm running HPD on a Virtual box.

I've executed the command from sandbox hdp SSH client: hdfs dfs -du -h / But this is obviously HDFS data usage.

12.2 M   /app-logs
1.5 G    /apps
0        /ats
860.9 K  /demo
724.4 M  /hdp
0        /livy2-recovery
0        /mapred
0        /mr-history
479.6 M  /ranger
176.6 K  /spark2-history
0        /tmp
4.0 G    /user
0        /webhdfs

Solution

  • Just treat this like any other disk almost full issue.

    Login to the sandbox. Run du -s /*/* to see what is using up disk space. I suspect it's probably the log files (under /var/log/*).