How can I get the train and test scores for each iteration of a MLPRegressor?

Question

This answer seems exactly what I need BUT for a regressor instead of a classifier. https://stackoverflow.com/a/46913459/9726897

I made very minor modifications to modified the code provided by sascha from link as shown below. I thought it would be fairly straightforward to use for my MLPRegressior... but I'm getting an error message I don't know how to fix Any help would be greatly appreciated:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.neural_network import MLPRegressor


estimator_reg = MLPRegressor(
    solver='adam',
    activation='relu',
    learning_rate='adaptive',
    learning_rate_init=.01,
    hidden_layer_sizes=[100],
    alpha=0.01,
    max_iter=1000,
    random_state=42,
    tol=0.0001,
    early_stopping=False,
    warm_start=True,
    beta_1=0.7,
    beta_2=0.98,
    epsilon=0.0000000001,
    verbose=10,
)

""" Home-made mini-batch learning
    -> not to be used in out-of-core setting!
"""
N_TRAIN_SAMPLES = train_data.shape[0]
N_EPOCHS = 25
N_BATCH = 128


scores_train = []
scores_test = []

# EPOCH
epoch = 0
while epoch < N_EPOCHS:
    print('epoch: ', epoch)
    # SHUFFLING
    random_perm = np.random.permutation(train_data.shape[0])
    mini_batch_index = 0
    while True:
        # MINI-BATCH
        indices = random_perm[mini_batch_index:mini_batch_index + N_BATCH]
        estimator_reg.partial_fit(train_data[indices], train_labels[indices])
        mini_batch_index += N_BATCH

        if mini_batch_index >= N_TRAIN_SAMPLES:
            break

    # SCORE TRAIN
    scores_train.append(estimator_reg.score(train_data, train_labels))

    # SCORE TEST
    scores_test.append(estimator_reg.score(test_data, test_labels))

    epoch += 1

""" Plot """
fig, ax = plt.subplots(2, sharex=True, sharey=True)
ax[0].plot(scores_train)
ax[0].set_title('Train')
ax[1].plot(scores_test)
ax[1].set_title('Test')
fig.suptitle("Accuracy over epochs", fontsize=14)
plt.show()

and I get this error:
KeyError Traceback (most recent call last)
in ()
---> 46 estimator_reg.partial_fit(train_data[indices], train_labels[indices])
.......
.......
KeyError: '[ 789 1493 353 33 1011 2029 1696 1649 653 1648 22 2477 2120 1000\n 2481 2448 1704 1962 2291 1995 2085 710 967 1839 461 504 1650 2166\n 584 513 676 1196 1621 2109 766 2012 1017 1636 1286 448 2049 1791\n 141 1168 1249 159 2061 2456 431 1799 2249 2379 1169 1044 1010 120\n 2503 316 1070 671 1005 2164 975 2371 811 1555 1193 1316 487 1867\n 1262 1395 135 2224 32 1509 2132 997 263 233 1614 2317 1432 49\n 1251 2227 2536 1955 359 650 2287 792 1900 606 763 1837 742 965\n 1190 53 910 2486 738 103 1965 99 1084 123 1061 806 384 2261\n 2284 2114 360 1075 1479 1446 455 2294 221 1856 979 1078 2106 189\n 2153 1183] not in index'

Answer 1

I guess that you have indexes that are not in the range (0,N_TRAIN_SAMPLES).
That may happen if you deleted or filtered some rows, or the index contained from the begining some numbers not in that range.

Try changing this line:

random_perm = np.random.permutation(train_data.shape[0])

into this:

random_perm = np.random.permutation(train_data.index.values)