Understanding the Implications of Scaling Test Data Using the Same Scalar Object as Training Data

Question

I am currently working on a machine learning project and have encountered a dilemma regarding the scaling of test data. I understand that when scaling features, we fit the scalar object using the training data and then transform both the training and test data using the same scalar object.

However, I have a concern regarding potential data leakage when scaling the test data. As the scalar object is based on the statistical properties (e.g., mean, standard deviation) calculated from the training data, I am unsure how accurately it can scale the test data without incorporating information from the test set.

Could someone please clarify whether there is a risk of data leakage when transforming the test data with the same scalar object used for the training data? If so, what would be the best approach to mitigate this risk and ensure a reliable evaluation of the model's performance?

I appreciate any insights or guidance from the community to help address my confusion and ensure proper scaling practices in my machine learning project.

Thank you in advance for your help and expertise.

Answer 1

When scaling your data, you must "learn" the scaling parameters (creating the scaler) only using your training dataset, just as you wrote. There is no leakage when you use the same scaler for your test set.

The only thing you should make sure of, in that context, is that you first split your data and create the scaler on the training set. make sure to not re-create the scaler when applying on the test set.

Learning from the training set is the same whether its the model's parameter or its the minimum and maximum of the distribution (or any other property).

Another thing to keep in mind is that if you want the values to be in some range, let's say [0,1] and you created a scaler using your training set. There still is a possibility that the there is some extreme value in the test set and your scaler won't map it into the same range. You can address this possible issue by forcing extreme values to be mapped to the edges of your range.

I hope this helps.