How does tf.multinomial work?

Question

How does tf.multinomial work? Here is stated that it "Draws samples from a multinomial distribution". What does that mean?

Answer 1

If you perform an experiment n times that can have only two outcomes (either success or failure, head or tail, etc.), then the number of times you obtain one of the two outcomes (success) is a binomial random variable.

In other words, If you perform an experiment that can have only two outcomes (either success or failure, head or tail, etc.), then a random variable that takes value 1 in case of success and value 0 in case of failure is a Bernoulli random variable.

---

If you perform an experiment n times that can have K outcomes (where K can be any natural number) and you denote by X_i the number of times that you obtain the i-th outcome, then the random vector X defined as

X = [X_1, X_2, X_3, ..., X_K]

is a multinomial random vector.

In other words, if you perform an experiment that can have K outcomes and you denote by X_i a random variable that takes value 1 if you obtain the i-th outcome and 0 otherwise, then the random vector X defined as

X = [X_1, X_2, X_3, ..., X_K]

is a Multinoulli random vector. In other words, when the i-th outcome is obtained, the i-th entry of the Multinoulli random vector X takes value 1, while all other entries take value 0.

So, a multinomial distribution can be seen as a sum of mutually independent Multinoulli random variables.

And the probabilities of the K possible outcomes will be denoted by

p_1, p_2, p_3, ..., p_K

---

An example in Tensorflow,

In [171]: isess = tf.InteractiveSession()

In [172]: prob = [[.1, .2, .7], [.3, .3, .4]]  # Shape [2, 3]
     ...: dist = tf.distributions.Multinomial(total_count=[4., 5], probs=prob)
     ...: 
     ...: counts = [[2., 1, 1], [3, 1, 1]]
     ...: isess.run(dist.prob(counts))  # Shape [2]
     ...: 
Out[172]: array([ 0.0168    ,  0.06479999], dtype=float32)

Note: The Multinomial is identical to the Binomial distribution when K = 2. For more detailed information please refer either tf.compat.v1.distributions.Multinomial or the latest docs of tensorflow_probability.distributions.Multinomial