I have to run many arff files in weka, and for each of them I have to run multiple classifiers- MLP, RandomForest,FURIA, etc., with different test options for each, and store each of their results. It is very time consuming to do each of them manually. I am looking for some method, or some script that can automate the process. I must confess that I am new to machine learning, and have no knowledge of scripting. I found this- https://sites.google.com/site/svaisipour/utilities/cs-for-biologist/weka-runner , and though I get how to manipulate the classifiers here, I cant understand how to change the test options from cross-validation to percentage-split.
Please let me know of any method that might help me with this. Any help will be deeply appreciated.
You can use the Weka Experimenter GUI for this.
A big dialog box will appear next.
To get run experiments with multiple algorithms, you must click on the "New" Button there, and then... oh well, ... this video about the experimenter will get you started.