How many principal components should I use in pattern classification?

Question

I am working on neural networks and I am currently creating a perceptron that will work as a classifier for a data set of images with faces. I am required to perform pca (principal component analysis) to my data set before dividing the samples into two different sets for training and testing. By doing this, I am lowering the dimensionality of the data and at the same time I am compressing the size of the images.

However, I am not a statistician and I have some problems defining the number of principal components to use for the pca method without any specific formula. My data set is an array of 4096x400, 400 being the number of the sample images and 4096 being their dimension. Is there a way to be more precise and accurate about the number of principal components to use during pca?

I am working on matlab so I am using princomp. Thank you in advance, any help will be highly appreciated.

Answer 1

Question: How many principal components should I use in pattern classification?

Answer: As low as possible.

When you apply PCA, you get number of principal components according to your data. Lets say you get 10 principal components from your data. You will control how much your variance are explained with principal components.

For example

  component  variance explained
  1          0.40
  2          0.25
  3          0.15
  4          0.10
  5          0.05
  6          0.01
  7          0.01
  8          0.01         
  9          0.01         
  10         0.01         

With this, you decide on cutoff number and train your classifier. In this example, as you can see first 4 principal components holds %90 of information. Your results may be good enough with only 4 principal components.

You may add 5th principal components, these 5 principal components will hold %95 of your information, and so on.

See an example with PCA and image data here