Search code examples
tensorflowconv-neural-networkword2vec

Word Embedding for Convolution Neural Network


I am trying to apply word2vec for convolution neural network. I am new with Tensorflow. Here is my code for pre-train layer.

W = tf.Variable(tf.constant(0.0, shape=[vocabulary_size, embedding_size]),
                trainable=False, name="W")
embedding_placeholder = tf.placeholder(tf.float32, [vocabulary_size, embedding_size])
embedding_init = W.assign(embedding_placeholder)
sess = tf.Session()
sess.run(embedding_init, feed_dict={embedding_placeholder: final_embeddings})

I think I should use embedding_lookup but not sure how to use it. I really appreace it someone could give some advice.

Thanks


Solution

  • You are on the right track. As embedding_lookup works under the assumption that words are represented as integer ids you need to transform your inputs vectors to comply with that. Furthermore, you need to make sure that your transformed words are correctly indexed into the embedding matrix. What I did was I used the information about the index-to-word-mapping generated from the embedding model (I used gensim for training my embeddings) to create a word-to-index lookup table that I subsequently used to transform my input vectors.