How to do model comparison with a dummy variable

Questions v5
5

Mh, I see. So if I got this right, I would

  • add yet another level of hierarchy to the model and
    introduce the the model probability as another variable (let’s say \mu), so the model becomes
    p(\mu, m, \theta, x) = p(\mu) \cdot p(m | \mu) \cdot p(\theta) \cdot p(x | \theta_m, m)
  • then marginalize out the model choice variable m analytically:
    p(\mu, \theta, x) = p(\mu) \cdot p(\theta) \cdot p(x | \theta, \mu)
    (where $p(x | \theta, \mu) is the weighted mixture likelihood that you mentioned)
  • and finally look at the posterior of \mu instead of the posterior of m.

Is that right?