Search code examples
linuxazurehadoophdfshortonworks-data-platform

How to find Hadoop hdfs directory on my system?


How to find Hadoop HDFS directory on my system? I need this to run following command -

hadoop dfs -copyFromLocal <local-dir> <hdfs-dir>

In this command I don't knon my hdfs-dir.

Not sure if its helpful or not but I ran following command and got this output -

 hdfs dfs -ls
-rw-r--r--   3 popeye hdfs  127162942 2016-04-01 19:47 .

In hdfs-site.xml, I found following entry -

<property>
      <name>dfs.datanode.data.dir</name>
      <value>/hadoop/hdfs/data</value>
      <final>true</final>
</property>

I tried to run following command but it gives error -

[root@sandbox try]# hdfs dfs -copyFromLocal 1987.csv /hadoop/hdfs/data
copyFromLocal: `/hadoop/hdfs/data': No such file or directory

FYI - I am doing all this on hortonworks sandbox on azure server.


Solution

  • Your approach is wrong or may be understanding is wrong

    dfs.datanode.data.dir, is where you want to store your data blocks

    If you type hdfs dfs -ls / you will get list of directories in hdfs. Then you can transfer files from local to hdfs using -copyFromLocal or -put to a particular directory or using -mkdir you can create new directory

    Refer below link for more information

    http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html