Search code examples
javahadoophdfshadoop2

“Wrong FS… expected: file:///” when trying to copyFromLocalFile from HDFS in Java

I am trying to copy abc.json from port/example_File/2017 to another location /port/example_File/2018 in HDFS, by below code

String exampleFile= "hdfs://port/example_File/2017/abc.json"
String targetFile="hdfs://port/example_File/2018"
hdfs.copyFromLocalFile(new Path(exampleFile),new Path(targetFile))

I am getting below exception

    org.jboss.resteasy.spi.UnhandledException: 
java.lang.IllegalArgumentException: Wrong FS:
 hdfs://port/example_File/2017/abc.json, expected: file:///

How to copy file from one location to another in HDFS

Thanks,

Solution

  • copyFromLocal is to copy files from Local Filesystem to HDFS.

    To copy files within an HDFS cluster, use FileUtil.copy() and specify same Filesystem for both srcFS and dstFs.

    Use DistCp when copying large number of files within a HDFS cluster or between two different HDFS clusters. Refer DistCp Java API here.