I want to predict some values with PCA in Python with sklearn.
I begin by taking in the relevant columns from the data and name them X for features and Y for features that need predicting.
Y = DF['Predict'].values
X = pd.DataFrame(data=scale(DF[X_cols]), columns=X_cols)
pca = PCA(n_components=NCOMPS) #NCOMPS=min(len(X_cols, Num_samples)
X_reduced = pd.DataFrame(pca.fit_transform(X),
columns=['PC%i' % i for i in range(NCOMPS)])
I've already plotted how well variance is explained by number of PC's, so I know I extracted the PC's alright.
I want to proceed by plotting the errors of the predicted Y based on the number of PC's.
How do I use what I have for prediction?
To top it all off I'd also want to add LOOCV, but I guess I'll reserve that for another question if I get stuck again.
LATER EDIT: I tried this but a dozen undo/redo's later I screwed it up and Spyder's edit history can no longer deliver me from this pain.
classifier = LogisticRegression()
total_err = []
for num_comps in range(1, NCOMPS):
classifier.fit(X_reduced, Y)
ypred = np.array(classifier.predict(X_reduced[:,:num_comps))
Y = np.array(Y)
total_err.append(abs(np.subtract(Y, ypred)))
Where's the mistake? Console says 'X has 2 features per sample; expecting 30'
You just need to pick a classifier/estimator and fit to your data.
from sklearn.datasets import load_iris
from sklearn.dimensionality_reduction import PCA
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
iris = load_iris()
X = iris.data
y = iris.target
pca = PCA()
X_reduced = pca.fit_transform(X)
rf = RandomForestClassifier()
X_train, X_test, y_train, y_test = train_test_split(X_reduced, y)
rf.fit(X_train, y_train)
rf.predict_proba(X_test)
array([[0. , 0.9, 0.1],
[0. , 0.8, 0.2],
[0.9, 0. , 0.1],
[0. , 0.2, 0.8],
[0. , 1. , 0. ],
[1. , 0. , 0. ],
[0. , 0.1, 0.9],
[0. , 0.3, 0.7],
[1. , 0. , 0. ],
[0. , 0. , 1. ],
[1. , 0. , 0. ],
[0.9, 0.1, 0. ],
[1. , 0. , 0. ],
[0. , 1. , 0. ],
[1. , 0. , 0. ],
[0. , 0.9, 0.1],
[0.9, 0. , 0.1],
[1. , 0. , 0. ],
[0. , 0.8, 0.2],
[1. , 0. , 0. ],
[0. , 0. , 1. ],
[0. , 0. , 1. ],
[0.1, 0.8, 0.1],
[0. , 0.7, 0.3],
[1. , 0. , 0. ],
[0.9, 0.1, 0. ],
[0. , 0.7, 0.3],
[0. , 0.1, 0.9],
[0. , 0.9, 0.1],
[0. , 0.9, 0.1],
[0. , 0. , 1. ],
[0. , 0. , 1. ],
[0. , 0.7, 0.3],
[0. , 0. , 1. ],
[0. , 0. , 1. ],
[1. , 0. , 0. ],
[1. , 0. , 0. ],
[1. , 0. , 0. ]])
rf.score(X_test, y_test)
0.9736842105263158
pca.inverse_transform(X_test)
array([[5.7, 2.8, 4.5, 1.3],
[5.7, 3. , 4.2, 1.2],
[5.4, 3.9, 1.3, 0.4],
[7.1, 3. , 5.9, 2.1],
[6.4, 2.9, 4.3, 1.3],
[5.7, 3.8, 1.7, 0.3],
[6.4, 3.1, 5.5, 1.8],
[7.7, 3. , 6.1, 2.3],
[4.8, 3. , 1.4, 0.3],
[6.9, 3.2, 5.7, 2.3],
[5.2, 4.1, 1.5, 0.1],
[4.6, 3.1, 1.5, 0.2],
[4.9, 3.1, 1.5, 0.1],
[6. , 2.2, 4. , 1. ],
[5. , 3.4, 1.5, 0.2],
[5.5, 2.4, 3.7, 1. ],
[5. , 3.5, 1.3, 0.3],
[5.5, 3.5, 1.3, 0.2],
[6. , 2.2, 5. , 1.5],
[4.8, 3. , 1.4, 0.1],
[6.9, 3.1, 5.4, 2.1],
[6.8, 3.2, 5.9, 2.3],
[5.6, 3. , 4.5, 1.5],
[5.6, 2.9, 3.6, 1.3],
[5.1, 3.8, 1.6, 0.2],
[4.3, 3. , 1.1, 0.1],
[6.6, 2.9, 4.6, 1.3],
[7.4, 2.8, 6.1, 1.9],
[5.6, 3. , 4.1, 1.3],
[5.8, 2.7, 4.1, 1. ],
[6.5, 3. , 5.2, 2. ],
[6.3, 2.9, 5.6, 1.8],
[6.9, 3.1, 4.9, 1.5],
[7.2, 3.2, 6. , 1.8],
[7.2, 3.6, 6.1, 2.5],
[5.4, 3.9, 1.7, 0.4],
[5.1, 3.5, 1.4, 0.2],
[5.8, 4. , 1.2, 0.2]])
y_test
array([1, 1, 0, 2, 1, 0, 2, 2, 0, 2, 0, 0, 0, 1, 0, 1, 0, 0, 2, 0, 2, 2,
1, 1, 0, 0, 1, 2, 1, 1, 2, 2, 1, 2, 2, 0, 0, 0])