Search code examples
machine-learningdeep-learningnlpartificial-intelligence

Can we use more than one target word for the CBOW method?

Say we have two target words for the same contexts:

  1. he is a VERY GOOD BOY.
  2. This is a VERY GOOD HAND-WRITING.

Here the VERY AND GOOD are contexts, whilst BOY and HAND-WRITING are two targets.

Can anyone explain how to do it?

Solution

  • This is an extreme case and it shouldn't happen in a normal corpus.

    In CBOW model, initialization of weights and training process determine the final result, and the training process depending on the context of target.

    Assuming that BOY and HAND-WRITING use the same initial weights and their context in 3-gram is completely equal, you can consider them as the same one target.

    If they have different initial weights, you would get two similar word representations after training, which is said that they are synonyms. This is what CBOW want: words who have similar context would have similar semantics.