Understanding a tensorflow confusion matrix for binary classification

Question

I have a binary classification problem that I'm trying to tackle in tensorflow. I'm using a simple multi layer perceptron. I'm trying to understand the confusion matrix I am getting. A sample output is:

 Epoch: 0001 cost=882.103631592   
 Epoch: 0002 cost=496.739675903   
 Epoch: 0003 cost=403.711282349   
 Epoch: 0004 cost=389.798379517   
 Epoch: 0005 cost=324.857388306     
 Optimization Finished!   
 Accuracy: 0.889306   
 CM=    
 [[797  260]  
 [   0 1071]]

The labels are AWAKE and NOT_AWAKE. It looks like from doing a one-hot encoding, I have [1,0] as AWAKE and [0,1] as NOT_AWAKE (I just save the array to a file and visually inspect).
How do I interpret the confusion matrix ? I believe this output:

 CM=    
 [[797  260]  
 [   0 1071]]

May be interpreted as:

             Pred: 0 | Pred: 1
 Actual 0:    797    |   260                
 Actual 1:    0      |   1071

Does [1,0] (one hot encoding for AWAKE) become row 1 in the confusion matrix? Most of the code to run the mlp is below.

# Parameters
learning_rate = 0.00001
training_epochs = 4
display_step = 1
keep_prob_training = 0.75

# Network Parameters
n_hidden_1 = 2048  # 1st layer number of neurons
n_hidden_2 = 2048  # 2nd layer number of neurons
n_input = 9  # channels
n_classes = 2  # total classes

print( "Some hyper params: training_epochs = %s,learning_rate = %f,keep_prob_training = %s, n_hidden_1 = %s,n_hidden_2 = %s" % ( training_epochs, learning_rate, keep_prob_training, n_hidden_1, n_hidden_2 ) )
print ( "Misc shape info: X_train.shape = %s, X_test.shape = %s, y_train.shape  = %s, y_test.shape = %s" % ( np.shape( X_train ), np.shape( X_test ), np.shape( y_train ), np.shape( y_test ) ) )

# tf Graph input
X = tf.placeholder( "float", [None, n_input] )
Y = tf.placeholder( "float", [None, n_classes] )
keep_prob = tf.placeholder( tf.float32 )
# placeholder for confusion matrix
y_ = tf.placeholder( tf.float32, shape = [None, 2] )


# Store layers weight & bias
weights = {
    'h1': tf.Variable( tf.random_normal( [n_input, n_hidden_1] ) ),
    'h2': tf.Variable( tf.random_normal( [n_hidden_1, n_hidden_2] ) ),
    'out': tf.Variable( tf.random_normal( [n_hidden_2, n_classes] ) )
}
biases = {
    'b1': tf.Variable( tf.random_normal( [n_hidden_1] ) ),
    'b2': tf.Variable( tf.random_normal( [n_hidden_2] ) ),
    'out': tf.Variable( tf.random_normal( [n_classes] ) )
}


# Create model
def multilayer_perceptron( x ):
    # Hidden fully connected layer
    layer_1 = tf.add( tf.matmul( x, weights['h1'] ), biases['b1'] )
    layer_1 = tf.nn.relu( layer_1 )
    # apply DropOut to hidden layer
    drop_out_1 = tf.nn.dropout( layer_1, keep_prob )
    # Hidden fully connected layer
    layer_2 = tf.add( tf.matmul( drop_out_1, weights['h2'] ), biases['b2'] )
    layer_2 = tf.nn.relu( layer_2 )
    drop_out_2 = tf.nn.dropout( layer_2, keep_prob )
    # Output fully connected layer with a neuron for each class
    out_layer = tf.matmul( drop_out_2, weights['out'] ) + biases['out']
    return out_layer

# Construct model
logits = multilayer_perceptron( X )

# obtain cm after training
confusion_matrix_tf = tf.confusion_matrix( tf.argmax( logits, 1 ), tf.argmax( y_, 1 ) )

# Define loss and optimizer
loss_op = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits( logits = logits, labels = Y ) )
optimizer = tf.train.AdamOptimizer( learning_rate )
train_op = optimizer.minimize( loss_op )

# Initializing the variables
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run( init )

    # Training cycle
    for epoch in range( training_epochs ):
        avg_cost = 0.

        # Run optimization op (backprop) and cost op (to get loss value)
        _, c = sess.run( [train_op, loss_op], feed_dict = {X: X_train, Y: y_train, keep_prob : keep_prob_training} )
        # Compute average loss
        # Display logs per epoch step
        if epoch % display_step == 0:
            print( "Epoch:", '%04d' % ( epoch + 1 )  , "cost={:.9f}".format( c ) )
    print( "Optimization Finished!" )

    # Test model
    pred = tf.nn.softmax( logits )  # Apply softmax to logits
    correct_prediction = tf.equal( tf.argmax( pred, 1 ), tf.argmax( Y, 1 ) )
    # Calculate accuracy
    accuracy = tf.reduce_mean( tf.cast( correct_prediction, "float" ) )
    print( "Accuracy:", accuracy.eval( {X: X_test, Y: y_test, keep_prob : 1.0} ) )

    cm = confusion_matrix_tf.eval( feed_dict = {X: X_train, y_: y_train, keep_prob: 1.0} )
    print( "CM=\n", cm ) 

Here is how I encode my labels:

    label_encoder = LabelEncoder()
    integer_encoded = label_encoder.fit_transform( df_combined['Label'] )

    # binary encode
    onehot_encoder = OneHotEncoder( sparse = False )
    integer_encoded = integer_encoded.reshape( len( integer_encoded ), 1 )
    all_y = onehot_encoder.fit_transform( integer_encoded )

Answer 1

Regarding the Tensorflow confusion matrix, your assumption on how it is interpreted is correct.

For example:

number of classes = 2
Predicted labels = [0,  1, 1, 1,  0, 0, 0, 0,  1, 1]
Actual labels    = [0,  1, 1, 1,  1, 1, 1, 1,  0, 0]

So your Tensorflow confusion matrix will be:

             Pred: 0 | Pred: 1
 Actual 0:    1      |   2                
 Actual 1:    4      |   3

Next, on is AWAKE interpreted as [0, 1] or [1, 0] depends upon what label have you assigned to AWAKE before you did one-hot encoding on it (You have not enclosed that part of code). For example, if you had assigned AWAKE as 0 and since there are only two classes totally, one-hot encoding will give you [1, 0].

Hope this answer helps you!