Search code examples
javaapachehadoopant

No FileSystem for scheme:hdfs and Class org.apache.hadoop.DistributedFileSystem not found

I want to upload a file to HDFS. I compiled my code using following jars as dependencies:

  • hadoop-auth-2.6.1.jar,
  • hadoop-common-2.6.1.jar and
  • hadoop-hdfs-2.6.1.jar,

My code:

enter image description here

I compiled it with Ant. But, it gave me this error: No FileSystem for scheme:hdfs.

Then I changed the code and compiled again: enter image description here

But now I got another error: Class org.apache.hdfs.DistributedFileSystem not found.

What's wrong? And what should I do?

Solution

  • DistributedFileSystem is part of hadoop-core.

    To fix this problem, you need to include hadoop-core-1.2.1.jar also (Note: I am using Maven for building):

    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.2.1</version>
</dependency>

    Overall, I am using following Maven dependencies:

    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.7.1</version>
</dependency>

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-core</artifactId>
    <version>2.7.1</version>
</dependency>

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.2.1</version>
</dependency>