Search code examples
pythontensorflowneural-network

How to predict the accuracy of input data after training a neural network


The essence of the problem: I wanted to understand the simplest neural network using a ready-made example from the Internet. I trained it, and then I don't understand how to test its validity on the input data of a conditional user. I found functions for model on the Internet: predict(), save(), loaded_model(). If save() and loaded_model() turned out and the folder '16_model' was created, then errors are thrown from predict(). Please tell me how to use it or how to test the neural network on input, and not on test data.

import keras
from keras.models import Sequential
from keras.layers import Dense
import numpy

numpy.random.seed(2)

dataset = numpy.loadtxt("diabet.csv", delimiter=",")
X, Y = dataset[:,0:8], dataset[:,8]

model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss="binary_crossentropy", optimizer="adam", metrics=['accuracy'])

model.fit(X, Y, epochs = 100, batch_size=10)

scores = model.evaluate(X, Y)
#model.save('16_model')
#model_loaded = keras.models.load_model('16_model')
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

Solution

  • You can split the dataset into training and testing part as below, and use those training set to train the model and test dataset to evaluate the model:

    import pandas as pd
    from sklearn.model_selection import train_test_split
    dataset =pd.read_csv("/content/../diabetes.csv")#, delimiter=",")
    dataset.head()
    X = dataset.drop('Outcome', 1)
    y = dataset['Outcome']
    Y.shape
    
    train_X, test_X, train_Y, test_Y=train_test_split(X,y, test_size=0.2)
    train_X.shape
    

    Then after defining and training the model with train dataset, you can evaluate the model with test dataset.

    model_loss, model_acc = model.evaluate(test_X, test_Y)
    
    print("Model Accuracy", model_acc*100)
    model.save('16_model')
    model_loaded = keras.models.load_model('16_model')
    

    Output:

    5/5 [==============================] - 0s 2ms/step - loss: 0.5637 - accuracy: 0.7208
    Model Accuracy 72.07792401313782
    INFO:tensorflow:Assets written to: 16_model/assets
    

    Now evaluating the model again after saving and loading back the same model which shows same accuracy as before of saving the model.

    model_loss, model_acc = model_loaded.evaluate(test_X, test_Y)
    

    Output:

    5/5 [==============================] - 0s 3ms/step - loss: 0.5637 - accuracy: 0.7208
    

    Here is Prediction Part:

    (This will show all the value within the range 0 to 1 due to sigmoid function used for binary classification)

    pred=model_loaded.predict(train_X[:2])
    print(pred))
    

    Output:

    [[0.925119  ]
     [0.45006576]]
    

    All Values from <0.5 will come under class 0 and rest all will be assumed as class 1.

    for p in pred:
        if p>=.5:
            pred_class=1
        else:
            pred_class=0
        print(pred_class)
    

    Output:

    1
    0
    

    You can verify this prediction with actual label:

    train_Y[:2]
    

    Output:

    369    1
    653    0
    Name: Outcome, dtype: int64