Bayes Factor ADVI

How do I compute Bayes factor for two different models when using ADVI?

For example, if I want to compare a two-state model with a three-state model.

Depending on how you compute the Bayes Factor, but if you are using sample based method (such as Savage-Dickey density ratio or Bridge sampling), you can sample from the ADVI to get a trace for each model and then use the trace to compute Bayes Factor.

Otherwise, since the ADVI provides a MvNormal approximation, which the marginal posterior likelihood is available, you can in theory compute the Bayes Factor directly.

But I think this is not a good way to compare two models, as Bayes Factor is sensitive to all sorts of things in your model, and ADVI usually is biased approximation of the true posterior. Mixing two flimsy things together sounds like a receipt for disaster.

1 Like