I'm facing an issue where 3 different classifiers, all trained on the same dataset (the sklearn iris dataset), are outputting the exact same accuracy scores and confusion matrices. I've emailed my professor and asked if this was something normal and if she had any advice if it wasn't and all she gave me was basically "it's not normal, go back and look at your code".
I've done a fair bit of looking at my code since then, and I can't seem to see what's going on. I'm hoping someone on here will be able to shed some light on it for me and I'll be able to learn something from this experience.
Here is my code:
# Dataset
from sklearn import datasets
# Data Preprocessing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
# Classifiers
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
# Performance Metrics
from sklearn.metrics import confusion_matrix, accuracy_score
if __name__ == '__main__':
# Read dataset into memory.
iris = datasets.load_iris()
# Extract independent and dependent variables into variables.
X = iris.data
y = iris.target
# Split training and test sets (70/30).
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Fit the scaler to the training set, and transform both the training and test sets dependent
# columns, which are all of them since none of the dependent variables contain categorical data.
ss = StandardScaler()
X_train = ss.fit_transform(X_train)
X_test = ss.transform(X_test)
# Create the classifiers.
dt_classifier = DecisionTreeClassifier(random_state=0)
svm_classifier = SVC(kernel='rbf', random_state=0)
lr_classifier = LogisticRegression(random_state=0)
# Fit the classifiers to the training data.
dt_classifier.fit(X_train, y_train)
svm_classifier.fit(X_train, y_train)
lr_classifier.fit(X_train, y_train)
# Predict using the now trained classifiers.
dt_y_pred = dt_classifier.predict(X_test)
svm_y_pred = svm_classifier.predict(X_test)
lr_y_pred = lr_classifier.predict(X_test)
# Create confusion matrices using the predicted results and the actual results from the test set.
dt_cm = confusion_matrix(y_test, dt_y_pred)
svm_cm = confusion_matrix(y_test, svm_y_pred)
lr_cm = confusion_matrix(y_test, lr_y_pred)
# Calculate accuracy scores using the predicted results and the actual results from the test set.
dt_score = accuracy_score(y_test, dt_y_pred)
svm_score = accuracy_score(y_test, svm_y_pred)
lr_score = accuracy_score(y_test, lr_y_pred)
# Print confusion matrices and accuracy scores for each classifier.
print('--- Decision Tree Classifier ---')
print(f'Confusion Matrix:\n{dt_cm}')
print(f'Accuracy Score:{dt_score}\n')
print('--- Support Vector Machine Classifier ---')
print(f'Confusion Matrix:\n{svm_cm}')
print(f'Accuracy Score:{svm_score}\n')
print('--- Logistic Regression Classifier ---')
print(f'Confusion Matrix:\n{lr_cm}')
print(f'Accuracy Score:{lr_score}')
Output:
--- Decision Tree Classifier ---
Confusion Matrix:
[[16 0 0]
[ 0 17 1]
[ 0 0 11]]
Accuracy Score:0.9777777777777777
--- Support Vector Machine Classifier ---
Confusion Matrix:
[[16 0 0]
[ 0 17 1]
[ 0 0 11]]
Accuracy Score:0.9777777777777777
--- Logistic Regression Classifier ---
Confusion Matrix:
[[16 0 0]
[ 0 17 1]
[ 0 0 11]]
Accuracy Score:0.9777777777777777
As you can see, the outputs for each different classifier are exactly the same. Any sort of help that anyone could give me would be greatly appreciated.
There is nothing wrong with your code.
Such similarities in results are not to be unexpected when:
Both these premises hold here. The iris data are notoriously easy to classify with modern ML algorithms (including the ones you use here); this, combined with the ridiculously small size of your test set (just 45 samples), make such results unsurprising.
In fact, simply by changing your data split to use a test_size=0.20, you will get a perfect accuracy of 1.0 from all 3 models.
Nothing to worry about.