I am trying to setup IGFS with Hadoop as the secondary storage. I have set my configuration as shown here but I keep getting NoClassDefFoundErrors. I have downloaded both binary distributions of Ignite and have tried building from source also but the dependencies are not included. hadoop-common-2.6.0.jar and ignite-hadoop-1.4.0.jar provided some of the dependencies but now I am getting a NoClassDefFoundError for org/apache/hadoop/mapred/JobConf which by my understanding is a deprecated class...
I have been following the instructions on the Apache Ignite website but this is as far as I've gotten.
What dependencies do I need for IgniteHadoopIgfsSecondaryFileSystem as the secondary storage?
It looks like the problem is that Ignite node does not have Hadoop libraries on the classpath. To fix that please try to do the following:
1) use "Hadoop Accelerator" edition of Ignite distribution (use -Dignite.edition=hadoop if you're building the distribution yourself).
2) Set HADOOP_HOME environment variable for the Ignite process if you're using Apache Hadoop distribution, or, if you use another distribution (HDP, Cloudera, BigTop, etc.) make sure /etc/default/hadoop file exists and has appropriate contents.
Alternatively, you can manually add necessary Hadoop dependencies to Ignite node classpath: these are dependencies of groupId "org.apache.hadoop" listed in file modules/hadoop/pom.xml . Currently they are: