I have done a Select attributes PCA in WEKA explorer, but I have troubles interpreting the output because new attribute output vector does not add up to 1.
My understanding is, given some attributes the PCA should return a smaller subset which is defined as new attributes that combine the old ones (Ex. given attributes A,B,C the PCA should return a new attribute vector 0,25A+0,5B+0,25C).
But what I get is:
−0.1669A +0.542B −0.4038C +0.4344D −0.3698E −0.3523F +0.0329G +0.0797H +0.4103I +0.4252J
This sums up to 0.6317 with negative signs or 2.8835 if considered as absolute values. Any ideas?
The weighting factors do not have to sum up to 1 because the final result does not have probability dimensions. The weight factors are so-to-say "similarity/contribution" factors. Thus, it is completely normal for an eigenvector to have a negative weight because it does not "contribute" to the final result.
I suggest you take a look at the following:
http://www.pages.drexel.edu/~sis26/Eigenface%20Tutorial.htm https://en.wikipedia.org/wiki/Eigenface
I consider face recognition/reconstruction from eigenfaces a good example for PCA analysis, because you have a visual idea of "what" an eigenvector is and how it is used
A good example of negative weights is the following:
If some of the people in the dataset are wearing glasses, this feature is going to be more strongly represented on one or more of the eigenfaces. In order to construct the face of a person who does not wear glasses, these eigenfaces are strongly irrelevant (likely to have a small or negative weight) and strongly relevant for persons wearing glasses.
If you go through more examples I am sure you will get it!