Is there a way to make weka use a balanced training set, where it only takes an equal number of instances for each class?
From the Classify panel choose metalearner "FilteredClassifier", and make its filter parameter "weka.filter.supervised.instance.ClassBalancer". Then select which classifier to use, i.e. J48, AdaBoost, etc.