Why is sklearn.metrics support value changing every time?

Question

I'm working on training a supervised learning keras model to categorize data into one of 3 categories. After training, I run this:

dataset = pandas.read_csv(filename, header=[0], encoding='utf-8-sig', sep=',')

# split X and Y (last column)
array = dataset.values
columns = array.shape[1] - 1
np.random.shuffle(array)
x_orig = array[:, 1:columns]
testy = array[:, columns]
columns -= 1

# normalize data
scaler = StandardScaler()
testx= scaler.fit_transform(x_orig)

#onehot
testy = to_categorical(testy)

# load weights
save_path = "[filepath]"
model = tf.keras.models.load_model(save_path)

# gets class breakdown
y_pred = model.predict(testx, verbose=1)
y_pred_bool = np.argmax(y_pred, axis=1)
y_true = np.argmax(testy, axis=1)
print(sklearn.metrics.precision_recall_fscore_support(y_true, y_pred))

sklearn.metrics.precision_recall_fscore_support prints, among other metrics, the support for each class. Per this link, support is the number of occurrences of each class in y_true, which is the true labels. https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html

My problem: each run, support is different. I'm using the same data, and support for each class always adds up the same (but different than the total in the file – which I also don’t understand), but the number per class differs.

As an example, one run might say [16870, 16299, 7807] and the next might say [17169, 15923, 7884]. They add up the same, but each class differs.

Since my data isn't changing between runs, I'd expect support to be identical every time. Am I wrong? If not, what's going on? I've tried googling, but didn't get any useful results.

Potentially useful information: when I run sklearn.metrics.classification_report, I have the same issue, and the numbers from that match the numbers from precision_recall_fscore_support.

Sidenote: unrelated to above question, but I couldn't google-fu an answer to this one either, I hope that's ok to include here. When I run model.evaluate, part of the printout is e.g. 74us/sample. What does us/sample mean?

Answer 1

Add:

np.random.seed(42)

before you shuffle the array at

np.random.shuffle(array)

The reason for this is without seeding np.shuffle will create a different result each time. Thus when you feed the array into the model it will return a different result. Seeding allows you to shuffle it the same each time, thus creating reproducible results.

Or you can not shuffle and get the same array each time to feed into the model. Either or both methods will ensure reproducibility within the model.