Search code examples
neural-networknormalizationpca

Pre-processing data: Should I regularize data before or after running a PCA?

I am pre-processing input data for a NN. To reduce the dimension of the input I am running a PCA. I want to normalize my data, should I do it before running the PCA or after ? ie on the raw-data/PCA inputs or on the PCA outputs/NN inputs ?

Solution

  • Have you tried both versions already? As far as I know PCA seeks for dependent featurevector colums to reduce information redundancy in your feature vectors. Regularization addresses the scaling and weighting of feature values. Having this in mind PCA should not be influenced by regularization because it should not change the dependency of feature values. Therefore I would apply PCA first and only regularize the shrinkened feature vectors.