Search code examples
mathprobabilityprobability-theoryprobability-distribution

Conditional mutual information notations

It might be a naive question but I am really struggling with this. I have looked at number of papers and articles that present the formula of the Mutual information and the Conditional Mutual information.. they usually write it in two ways :

  1. first formula

and

  1. second formula

Are those formulas actually equivalent ? and does PX(x) actually means the same as P(x) ?

Solution

  • PX(x) is indeed the same as P(x). Both of them represent the probability P(X=x). These two formulas are equivalent. You can move from the second to the first by using the following law: P(A,B)/P(B)=P(A|B). Then applying this law on the second formula we get 

    p(x,y|z)/[p(x|z)p(y|z)]=[p(x,y,z)/p(z)]/[{p(x,z)/p(z)}*{p(y,z)/p(z)}]=
=[p(x,y,z)*p(z)]/[p(x,z)*p(y,z)]

    The last transition follows from the cancellation of one of the p(z). And the result gives you the first formula.