How to update posterior distribution in Gibbs Sampling?

Question

All, I'd like to estimate the parameter P ,Q whose prior distribution is

P ~ N (A,B)

Q ~ N (C,D)

Then, I find the full conditional distribution of P,Q are

P|Q ~ N (A*,B*)

Q|P ~ N (C*,D*)

where A* is a function of A , B , Q [A*=f(A,B,Q)]

B* is a function of A , B , Q [B*=f(A,B,Q)]

Hence, in Gibbs updating step,

1. [First iteration]

   • update P_0 to P_1 (had information of A,B,Q_0 & get A* _1, B* _1)
   • update Q_0 to Q_1 (had information of C,D,P_0 & get C* _1, D* _1) (the subscript denote the sample from nth iteration; 0 is initial value)

2. [Second iteration] my question is: am I going to

   • update P_1 to P_2 (using information of A , B , Q_1 to get A* _2 , B* _2) OR
   • update P_1 to P_2 (using information of A* _1,B* _1,Q_1 to get A* _2, B* _2)

In other words, do we use the same prior in every Gibbs step or we use previous step estimation result as our prior? I knew one of the concept of Gibbs is to update each parameter isolate, so I am going to use the information of Q_1 to update P_2. How about prior?

Answer 1

Use the same prior at every step. Do not alter the prior parameters after you have begun or you will not be performing Bayesian inference.