Search code examples
pythongensimdoc2vec

Finding the distance between 'Doctag' and 'infer_vector' with Gensim Doc2Vec?


Using Gensim's Doc2Vec how would I find the distance between a Doctag and an infer_vector()?

Many thanks


Solution

  • Doctag is the internal name for the keys to doc-vectors. The result of an infer_vector() operation is a vector. So as you've literally asked, these aren't comparable.

    You could ask a model for a known doc-vector, by its doc-tag key that was supplied during training, via model.docvecs[doctag]. That would be comparable to the result of an infer_vector() call.

    With two vectors in hand, you can use scipy routines to calculate various kinds of distance. For example:

    import scipy.spatial.distance.cosine as cosine_distance
    vec_by_doctag = model.docvecs["doc0007"]
    vec_by_inference = model.infer_vector(['a', 'cat', 'was', 'in', 'a', 'hat'])
    dist = cosine_distance(vec_by_doctag, vec_by_inference)
    

    You can also look at how gensim's Doc2VecKeyedVectors does similarity/distance between vectors that are known (by their doctag key names) inside a model, in its similarity() and distance() functions, at:

    https://github.com/RaRe-Technologies/gensim/blob/ca0dcaa1eca8b1764f6456adac5719309e0d8e6d/gensim/models/keyedvectors.py#L1701

    https://github.com/RaRe-Technologies/gensim/blob/ca0dcaa1eca8b1764f6456adac5719309e0d8e6d/gensim/models/keyedvectors.py#L1743