Search code examples
pythontensorflowrecurrent-neural-networksoftmax

Sampled Softmax Loss in Tensorflow with RNN - Dimensions mismatch problem

I have an implementation of a Many-to-one RNN with variable sequence length (a sentence classification problem)

I am trying to implement a sampled softmax loss since I have 500 classes and want to speed up the training.

The following are my input parameters shapes

WLast.shape
TensorShape([Dimension(500), Dimension(500)])

bLast.shape 
TensorShape([Dimension(500)])

labels.shape
TensorShape([Dimension(None), Dimension(500)])

pred_out.shape
TensorShape([Dimension(None), Dimension(500)])

Pred_out is the last prediction from the RNN

Problem is that when I run:

cost = tf.nn.sampled_softmax_loss(WLast,bLast,labels,pred_out,10,500)

it gives me this error:

InvalidArgumentError: Dimension must be 1 but is 500 for 'sampled_softmax_loss/ComputeAccidentalHits' (op: 'ComputeAccidentalHits') with input shapes: [?,500], [10].

I don't understand, the shapes matches the arguments of the function, does someone know what could I be doing wrong?

Thanks in advance!

Solution

  • I found this implementation: https://github.com/olirice/sampled_softmax_loss and solved the problem by reshaping the labels

    labels = tf.reshape(tf.argmax(labels, 1), [-1,1])