PCA for Recurrent Neural Networks (LSTM) - Shall I use PCA for target variables too?

Question

I have a seasonal timeseries dataset containing 3 target variables and n feature variables. I am trying to apply a PCA algorithm before feeding the data to a simple LSTM. The operations I do are the following:

1. Split train - validation - test
2. Standard scaler (force mean=0 & std=1) of the train dataset (including target and features)
3. Apply PCA for only features of the train dataset
4. Transform through the PCA matrix in step 3 the feature variables from validation and target
5. Where I get lost: What to do with target's validation and target's test variables?
6. ... more neural networks pre-processing and building the architecture of the LSTM

My question is: How do I scale / normalize the target variables? Through a PCA too?, through any independent scaler (standard, mapminmax, etc.)? If I leave the original target values I got overfitting in my LSTM.

The most disappointing is that without the PCA, the LSTM I've build is showing no overfitting

Thanks a lot for your help!

Answer 1

I know this comes late... As far as I know, you should not apply PCA to the target variables. PCA is used in a way to reduce dimensionality on the feature variables. As you have applied the PCA transformation trained with the Train dataset, you can do the same with the used Scaler.