Search code examples
nlpword-embeddingbert-language-model

BERT embeddings for entire sentences vs. verbs


First off, I am drawing upon assumption that majority of the semantic value of the sentence is mediated by verbs that connect the subject and the object of said verb. I am aware that I simplify a bit here as there can be multiple verbs and such. But abstracting from that, I'd be curious whether it applies that a BERT generated embedding vector for entire sentence, say "Two halves make a whole" would be measurably more similar to an embedding vector for a singular verb like "make" as compared to say a vector for verb "eat".


Solution

  • To answer your "question", if I were you I would try to do a practical test. For an easy way to use bert for sentence embeddings, check this repo: it is summarily simple to use.

    Once you have the embedding vectors, you can use any similarity function to validate your hypothesis.

    However for what is my (limited) experience, I think that the vector of "make" is more similar than that of "eat" also only because "make" is present in the other sentence and therefore contributes to the ambedding of the sentence.