Random Forest Classifier Removing Features using Top-N Features Method
I am trying to predict the winner of an NBA game using a random forest classifier. I have sought to remove and modify my list of features so that I can increase accuracy and decrease noise.
I implemented the solution found here: https://datascience.stackexchange.com/questions/57697/decision-trees-should-we-discard-low-importance-features, where I would loop through the top N most important features and plot out the resulting accuracy. After all my features have gone through that loop, I'm left with a plot that looks like this:
As you can see, the resulting graph is kind of all over the place. Do I remove the features that have a negative slope? Or what's the threshold to removing features? Is there a better way to calculate noise? How would I get the most accurate model given that I have so many features with such a variable impact on my model accuracy on training data?
As a starting point, you could try some feature selection techniques that are easier to understand. This is what I would try based on the small subset of techniques that I am familiar and comfortable with...
If you have continuous variables, plot a correlation matrix and remove highly correlated features to eliminate multicollinearity. If your features are categorical, you could try ANOVA. If you have a large number of features, a small sample size, and nonlinear relationships between features, you could investigate dimensionality reduction techniques like PCA.
- Importing a Model with Scikit Learn on Vertex
- Should nested modules with shared weights be an nn.Module object parameter or not?
- Convert array of indices to one-hot encoded array in NumPy
- All intermediate steps should be transformers and implement fit and transform
- How do I crop the solar panels captured by drone?
- How to concatenate a tensor to a keras layer along batch (without specifying batch size)?
- How do I import KerasRegressor in Google Colab?
- Text Image Binary Classifier without deep learning or tesseract
- What is the problem with my implementation of the cross-entropy function?
- `CrossEntropyLoss()` in PyTorch
- How to use Machine Learning to find the pattern customer profile?
- Why is ReLU a non-linear activation function?
- Extracting an information from web page by machine learning
- How to implement the Softmax function in Python?
- Does it make sense to use Conda + Poetry?
- ConvergenceWarning: lbfgs failed to converge (status=1): STOP: TOTAL NO. of ITERATIONS REACHED LIMIT
- Keras model not predicting values in the Test set
- Avoiding vanishing gradient in deep neural networks
- What is the best choice for an activation function in case of small sized neural networks
- SGD(Stochastic Gradient Descent) vs Backpropagation
- Stochastic Gradient Descent(SGD) vs Mini-batch size 1
- Question in one scikit ML example, "MNIST classification using multinomial logistic + L1"
- rpart() decision tree fails to generate splits (decision tree with only one node (the root node))
- AWS SageMaker Project template creation failing
- Resnet inconsistency between train and eval mode
- What is an Epoch in Neural Networks Training
- Difference between Generative, Discriminating and Parametric, Nonparametric Algorithm/Model
- Generate text with a trained character level LSTM model
- (MNIST - GAN) Discriminator and Generator error dropping close to zero after first iteration
- Loss functions in GANs