Hi All,
I just have a theoretical question about the approximation of model evidence.
If we use conventional MCMC methods, we can leverage information criteria such as BIC, and WAIC to compare each model and to draw corresponding model weights for each model.
But, if we move to variational inference, we rather use and optimize ELBO. And, ELBO is also an approximation of model evidence. In this case, do you think it is reasonable to compare two models’ ELBO to estimate the model weight of each model?
I personally think it may be a reasonable approach as they are all “approximations” of model evidence even for ELBO… but I cannot find any reference regarding this issue. I want to listen to your opinions about it.
Thanks,
Jinyoung