I am using a Siamese neural network to learn similarity between text.
Here is a SNN network I created for this task: it feeds two inputs into a Bidirectional LSTM, which shares/updates weights, and then produces two outputs. The distance between these two outputs is then calculated.
input_1 = Input(shape=(max_len,))
input_2 = Input(shape=(max_len,))
lstm_layer = Bidirectional(LSTM(50, dropout=0.2, recurrent_dropout=0.2)) # Won't work on GPU
embeddings_initializer = Constant(embed_matrix)
emb = Embedding(len(tokenizer.word_index)+1,
embedding_dim,
embeddings_initializer=embeddings_initializer,
input_length=max_len,
weights=[embed_matrix],
trainable=True)
e1 = emb(input_1)
x1 = lstm_layer(e1)
e2 = emb(input_2)
x2 = lstm_layer(e2)
mhd = lambda x: exponent_neg_cosine_distance(x[0], x[1])
merged = Lambda(function=mhd, output_shape=lambda x: x[0], name='cosine_distance')([x1, x2])
preds = Dense(1, activation='sigmoid')(merged)
model = Model(inputs=[input_1, input_2], outputs=preds)
model.compile(loss = "binary_crossentropy", metrics=['acc'], optimizer = optimizer)
However, I read recently that using triplet loss could improve my SNN. This is an example of a SNN that makes use of triplet loss for similarity learning:
embedding_model = tf.keras.models.Sequential([
tf.keras.Bidirectional(LSTM(50, dropout=0.2, recurrent_dropout=0.2))
tf.keras.layers.Dense(64, activation='relu', input_shape=(784,)),
tf.keras.layers.Dense(emb_size, activation='sigmoid')
])
input_anchor = tf.keras.layers.Input(shape=(784,))
input_positive = tf.keras.layers.Input(shape=(784,))
input_negative = tf.keras.layers.Input(shape=(784,))
embedding_anchor = embedding_model(input_anchor)
embedding_positive = embedding_model(input_positive)
embedding_negative = embedding_model(input_negative)
output = tf.keras.layers.concatenate([embedding_anchor, embedding_positive, embedding_negative], axis=1)
net = tf.keras.models.Model([input_anchor, input_positive, input_negative], output)
net.summary()
net.compile(loss=triplet_loss, optimizer=adam_optim)
def triplet_loss(y_true, y_pred, alpha = 0.4):
"""
Implementation of the triplet loss function
Arguments:
y_true -- true labels, required when you define a loss in Keras, you don't need it in this function.
y_pred -- python list containing three objects:
anchor -- the encodings for the anchor data
positive -- the encodings for the positive data (similar to anchor)
negative -- the encodings for the negative data (different from anchor)
Returns:
loss -- real number, value of the loss
"""
print('y_pred.shape = ',y_pred)
total_lenght = y_pred.shape.as_list()[-1]
# print('total_lenght=', total_lenght)
# total_lenght =12
anchor = y_pred[:,0:int(total_lenght*1/3)]
positive = y_pred[:,int(total_lenght*1/3):int(total_lenght*2/3)]
negative = y_pred[:,int(total_lenght*2/3):int(total_lenght*3/3)]
# distance between the anchor and the positive
pos_dist = K.sum(K.square(anchor-positive),axis=1)
# distance between the anchor and the negative
neg_dist = K.sum(K.square(anchor-negative),axis=1)
# compute loss
basic_loss = pos_dist-neg_dist+alpha
loss = K.maximum(basic_loss,0.0)
return loss
My confusion lies in the SNN network with the triplet loss. How is the distance between the three outputs calculated?
In the first SNN code chunk I included, this line merged = Lambda(function=mhd, output_shape=lambda x: x[0], name='cosine_distance')([x1, x2]) is calculating the distance between the two vectors.
But in the second SNN, I don't see where/if distance between the 3 vectors is calculated. If no distance calculation is necessary, why is that the case?
I'm not quite sure why you concatenated the three embedding vectors in the output. I suggest you peruse the document at https://keras.io/examples/vision/siamese_network/.
There, you'll find the below code snippet:
class DistanceLayer(layers.Layer):
"""
This layer is responsible for computing the distance between the anchor
embedding and the positive embedding, and the anchor embedding and the
negative embedding.
"""
def __init__(self, **kwargs):
super().__init__(**kwargs)
def call(self, anchor, positive, negative):
ap_distance = tf.reduce_sum(tf.square(anchor - positive), -1)
an_distance = tf.reduce_sum(tf.square(anchor - negative), -1)
return (ap_distance, an_distance)
anchor_input = layers.Input(name="anchor", shape=target_shape + (3,))
positive_input = layers.Input(name="positive", shape=target_shape + (3,))
negative_input = layers.Input(name="negative", shape=target_shape + (3,))
distances = DistanceLayer()(
embedding(resnet.preprocess_input(anchor_input)),
embedding(resnet.preprocess_input(positive_input)),
embedding(resnet.preprocess_input(negative_input)),
)
siamese_network = Model(
inputs=[anchor_input, positive_input, negative_input], outputs=distances
)
As you can see, they send the embeddings to the DistanceLayer class in which positive and negative distances are computed, and then returned as a tuple which is to be placed in the output of the model.