How can I process Persian texts using Rapid Miner?
I am working on a persian classification project. Persian texts is very similar to arabic texts. when I use Tokenize, it does not show any word in its wordlist page and in Example Set Page, The Image below will be shown:
I need to classify persian text to some category, but I dont know How?.
I Follow some steps like this:
1- Read Excel(using Read Excel component) dataset with 2 column => col1:persian Text ,col2: Category
2- I use Set role component to labeling data
3- I use Process Documents from Data component containing :(Tokenize(with any mode not change anythings) and Filter Token(min:5,max:25) inside it)
4- Then I use Cross Validation Component to train with SVM or Basian and in test mode to get performance.
The program runs correctly and performance is not bad for e.g accuracy is 50% but I think my work is Wrong.
Any help would be appreciated.
first, make sure your text data have UTF-8 encoding and if u use filter tokens(by length) 5 is too much for minimum try 2 or at least 3 also, I recommend using Filter Stopwords (Dictionary) operator and the dictionary should have Persian stopwords in each line hope it will help u
- NameError: name 'plot_confusion_matrix' is not defined
- Why do the sensitivity (recall) values differ between classification_report and precision_recall_fscore_support in a loop?
- How to compute sens slope with yearly data frame with value lower than 3?
- Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative
- Loss remains unchanged in a multiclass classification deep model
- AttributeError: 'Flags' object has no attribute 'c_contiguous'
- How to deal with columns in KNN?
- Why LightGBM with 'objective': 'binary' donot return binary value 0 and 1 when call method predict?
- Why am I seeing Index error in this Python script?
- How to use Dynamic Time warping with kNN in python
- Using sklearn for hierarchical classification
- ValueError: Arguments `target` and `output` must have the same shape. Received: target.shape=(None, 512), output.shape=(None, 3)
- Controlling the threshold in Logistic Regression in Scikit Learn
- What is 'mean' f1 score in machine learning?
- What are advantages of Artificial Neural Networks over Support Vector Machines?
- Classification report with Nested Cross Validation in SKlearn (Average/Individual values)
- Gender detection and age classification using deep learning
- XGBoost XGBClassifier Defaults in Python
- Difference between logistic regression and softmax regression
- Theory - How to calculate the weights and bias for a preceptron representing a 2D area
- What is the difference between a generative and a discriminative algorithm?
- Getting the accuracy for multi-label prediction in scikit-learn
- How to choose the right machine-learning classifer
- The right way of using SMOTE in Classification Problems
- sklearn ComplementNB: only class 0 predictions for perfectly seperable data
- Classification techniques for continuous arrays as inputs and scalar categorical variable as output
- Stacking ensemble of classifiers in a chain
- Classification using Graph Neural Network
- How does persisting the model increase accuracy?
- All models fail in a binary classification machine-learning task with tidymodels and XGBoost