I'm using HDP for self-study to learn Big data basics. Today I've faced the following: HDFS disk usage is 91%. With Non DFS Used 31.2 GB / 41.6 GB (74.96%).
What exactly should I do to free disk space? Is it possible to do from sandbox hdp SSH client? I'm running HPD on a Virtual box.
I've executed the command from sandbox hdp SSH client: hdfs dfs -du -h /
But this is obviously HDFS data usage.
12.2 M /app-logs
1.5 G /apps
0 /ats
860.9 K /demo
724.4 M /hdp
0 /livy2-recovery
0 /mapred
0 /mr-history
479.6 M /ranger
176.6 K /spark2-history
0 /tmp
4.0 G /user
0 /webhdfs
Just treat this like any other disk almost full issue.
Login to the sandbox. Run du -s /*/*
to see what is using up disk space. I suspect it's probably the log files (under /var/log/*
).