Search code examples
apache-storm

Storm not reading from HDFS (HdfsSpout not recognized)

Am using storm 1.0.x but when I try to use the below

// Instantiate spout to read text files
        HdfsSpout textReaderSpout = newHdfsSpout().setReaderType("text")
                                                  .withOutputFields(TextFileReader.defaultFields)
                                                  .setHdfsUri("hdfs://localhost:54310")  // reqd
                                                  .setSourceDir("/data/in")              // reqd
                                                  .setArchiveDir("/data/done")           // reqd
                                                  .setBadFilesDir("/data/badfiles");     // required

I get HdfsSpout cannot be resolved to a type

is this feature available in specific versions or what could be the problem ?

Solution

  • I would guess you don't have storm-hdfs on the classpath. Check your pom (or equivalent for whatever dependency management system you use), and make sure storm-hdfs is declared as a dependency.