How to modify the cost function of Neural Network in python

Question

I need to make my own cost function for a neural network in Python using the packages and libraries. For instance, I want to make a cost function that is a function of the output of one of the hidden layers.

Keras and MLP from scikit-learn does not allow that. Any better package?

In Keras, you can only have a modified cost function when it is a function of predicted y and actual y. I need more flexibilty.

Answer 1

You can create an auxiliry output, for example:

import tensorflow as tf

inp = tf.keras.layers.Input(...)
x1 = tf.keras.layers.Dense(..)(inp)
x2 = tf.keras.layers.Dense(...)(x1)

model = tf.keras.Model(inp, [x1, x2])

model.compile(loss=['loss_for_x1', 'loss_for_x2'], 
              optimizer='rmsprop',
              loss_weights=[1., 1.]) # How many the loss function influences 

You can thing of this as applying a loss directly to first layer.