Search code examples
hadoopmahout

debugging a mahout logistic regression

I am new to mahout.. And I am trying out the standard "donut" example listed here:

http://imiloainf.wordpress.com/2011/11/02/mahout-logistic-regression/

So this example works like a charm.

But when I try to implement it on my dataset (whcih is a huge dataset) it doesnt works.

The dataset is in one csv file.. everything is same except it has a lot of features (~100) and is 1TB file.

I am getting this error.

 bin/mahout trainlogistic  --input /path/mahout_input/complete/input.csv \
--output mahoutmodel --target default --categories 2 --predictors O1 E1 I1 \
 --types numeric --features 30 --passes 100 --rate 50
Running on hadoop, using HADOOP_HOME=/opt/mapr/hadoop/hadoop-0.20.2
No HADOOP_CONF_DIR set, using /opt/mapr/hadoop/hadoop-0.20.2/conf
Exception in thread "main" java.lang.NullPointerException
        at org.apache.mahout.classifier.sgd.CsvRecordFactory.firstLine(CsvRecordFactory.java:167)
        at org.apache.mahout.classifier.sgd.TrainLogistic.main(TrainLogistic.java:75)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:187)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

What am i doing wrong?

How do you debug this.. what is the error??

Thanks

Solution

  • My guess is your input doesn't exist or is empty. I'd check that /path/mahout_input/complete/input.csv is really what you mean.