I am trying to understand a code in word2vec project. The file I am referring to is word2vec.c. The snippet of the code is :
#define EXP_TABLE_SIZE 1000
#define MAX_EXP 6
//...snip...
expTable = (real *)malloc((EXP_TABLE_SIZE + 1) * sizeof(real));
for (i = 0; i < EXP_TABLE_SIZE; i++) {
// Precompute the exp() table
expTable[i] = exp((i / (real)EXP_TABLE_SIZE * 2 - 1) * MAX_EXP);
// Precompute f(x) = x / (x + 1)
expTable[i] = expTable[i] / (expTable[i] + 1);
}
//...snip...
It is not clear what is the benefit of precomputing these values. Could someone explain?
The table holds the values of exp
for arguments in the range -6
to 6
. The function is sampled at 1001 points.
The following code in the same source file word2vec.c
uses this table to calculate exponents:
...
if (f <= -MAX_EXP) continue;
else if (f >= MAX_EXP) continue;
else f = expTable[(int)((f + MAX_EXP) * (EXP_TABLE_SIZE / MAX_EXP / 2))];
...
(So if you wonder what value there is at the first cell of the table - it's exp(-6)
)