I have a model which has two sets of observed data, e.g.X and Y. In the model, \Theta is the set of free parameters. If one has a similar setting as this paper in section “CESNA hyperparameter settings” and wants to use the bayesian inference to compute the posterior of F (for instance the parameter given in the example), which kind of prior should put over \alpha (scaling parameter or weight of the likelihoods) in order the adjust the magnitude of two likelihoods? when the likelihoods are the normal distributions and the posterior and prior over the free parameters of the model are conjugate should the conjugacy be taken into account for the prior over \alpha? In my model, one part of observational data would have greater effect with its likelihood on the sampling than the other. I am confused whether it is the only way I can tackle the sampling problem of scaling two likelihoods? I’ll appreciate for any suggestion.
I dont know much about your application and your model, but from a quick read of the paper, seems the scaling \alpha is something between 0 and 1 (reminds me a lot about mixture model, but here they are different observed), I think you can put a Beta distribution as prior on it, and you need to decide the shape of the Beta distribution using some prior information you have.
In the first integral we can marginalize over W and in the second one over all F' but each of these two integral corresponds to a likelihood and the computed values of them have at least one oder of magnitude difference to the other. How can I define a scale parameter in this model and if the likelihoods and priors over W are normal distribution, would it be a problem to have a beta distribution as the prior over \alpha parameter in order to keep the conjugacy in this model?
It is likely will break the conjugacy - Not sure what would be the best way to deal with the double integral, these are usually quite tricky. Why not sample from your model? Then you dont need to worry about conjugacy.