In LDA model generates different topics everytime i train on the same corpus , by setting the np.random.seed(0), the LDA model will always be initialized and trained in exactly the same way.
Is it the same for the Word2Vec models from gensim? By setting the random seed to a constant, would the different run on the same dataset produce the same model?
But strangely, it's already giving me the same vector at different instances.
>>> from nltk.corpus import brown
>>> from gensim.models import Word2Vec
>>> sentences = brown.sents()[:100]
>>> model = Word2Vec(sentences, size=10, window=5, min_count=5, workers=4)
>>> model[word0]
array([ 0.04985042, 0.02882229, -0.03625415, -0.03165979, 0.06049283,
0.01207791, 0.04722737, 0.01984878, -0.03026265, 0.04485954], dtype=float32)
>>> model = Word2Vec(sentences, size=10, window=5, min_count=5, workers=4)
>>> model[word0]
array([ 0.04985042, 0.02882229, -0.03625415, -0.03165979, 0.06049283,
0.01207791, 0.04722737, 0.01984878, -0.03026265, 0.04485954], dtype=float32)
>>> model = Word2Vec(sentences, size=20, window=5, min_count=5, workers=4)
>>> model[word0]
array([ 0.02596745, 0.01475067, -0.01839622, -0.01587902, 0.03079717,
0.00586761, 0.02367715, 0.00930568, -0.01521437, 0.02213679,
0.01043982, -0.00625582, 0.00173071, -0.00235749, 0.01309298,
0.00710233, -0.02270884, -0.01477827, 0.01166443, 0.00283862], dtype=float32)
>>> model = Word2Vec(sentences, size=20, window=5, min_count=5, workers=4)
>>> model[word0]
array([ 0.02596745, 0.01475067, -0.01839622, -0.01587902, 0.03079717,
0.00586761, 0.02367715, 0.00930568, -0.01521437, 0.02213679,
0.01043982, -0.00625582, 0.00173071, -0.00235749, 0.01309298,
0.00710233, -0.02270884, -0.01477827, 0.01166443, 0.00283862], dtype=float32)
>>> exit()
alvas@ubi:~$ python
Python 2.7.11 (default, Dec 15 2015, 16:46:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from nltk.corpus import brown
>>> from gensim.models import Word2Vec
>>> sentences = brown.sents()[:100]
>>> model = Word2Vec(sentences, size=10, window=5, min_count=5, workers=4)
>>> word0 = sentences[0][0]
>>> model[word0]
array([ 0.04985042, 0.02882229, -0.03625415, -0.03165979, 0.06049283,
0.01207791, 0.04722737, 0.01984878, -0.03026265, 0.04485954], dtype=float32)
>>> model = Word2Vec(sentences, size=20, window=5, min_count=5, workers=4)
>>> model[word0]
array([ 0.02596745, 0.01475067, -0.01839622, -0.01587902, 0.03079717,
0.00586761, 0.02367715, 0.00930568, -0.01521437, 0.02213679,
0.01043982, -0.00625582, 0.00173071, -0.00235749, 0.01309298,
0.00710233, -0.02270884, -0.01477827, 0.01166443, 0.00283862], dtype=float32)
Is it true then that the default random seed is fixed? If so, what is the default random seed number? Or is it because I'm testing on a small dataset?
If it's true that the the random seed is fixed and different runs on the same data returns the same vectors, a link to a canonical code or documentation would be much appreciated.
Yes, default random seed is fixed to 1, as described by the author in https://radimrehurek.com/gensim/models/word2vec.html. Vectors for each word are initialised using a hash of the concatenation of word + str(seed).
Hashing function used, however, is Python’s rudimentary built in hash function and can produce different results if two machines differ in
Above list is not exhaustive. Does it cover your question though?
EDIT
If you want to ensure consistency, you can provide your own hashing function as an argument in word2vec
A very simple (and bad) example would be:
def hash(astring):
return ord(astring[0])
model = Word2Vec(sentences, size=10, window=5, min_count=5, workers=4, hashfxn=hash)
print model[sentences[0][0]]