Search code examples
neural-networktime-seriesregressionpredictionscaling

Unscale predictions back to original form

This is a regression problem.

The shape of my training is: (417, 5) and the test data shape is: (105, 5). I do scaling for both using the following code:

from sklearn import preprocessing
import sklearn
from sklearn.preprocessing import MinMaxScaler
#Scale train
scaler = preprocessing.MinMaxScaler()
train_df = scaler.fit_transform(train_df)
train_df = pd.DataFrame(train_df)

#Scale test
test_df = scaler.fit_transform(test_df)
test_df = pd.DataFrame(test_df)

First four rows of training data after scaling look like below:

enter image description here

while '4' is the dependent variable and the rest are independent variables.

After training using deep neural network, I get predictions in scaled form. I try to unscale predictions using the following code:

scaler.inverse_transform(y_pred_dnn)

while predictions are stored in y_pred_dnn

But I get the following error:

ValueError: non-broadcastable output operand with shape (105,1) doesn't match the broadcast shape (105,5)

How do I debug the problem?

Thanks

Solution

  • you can solve this by separating out y before scaling. You dont need to scale y for prediction. So try:

    y_train, y_test = train_df.iloc[:, 4], test_df.iloc[:, 4]
X_train, X_test = train_df.iloc[:, 1:4], test_df.iloc[:, 1:4]

    After this you do te scaling on X part only and you wont need any inverse scaling