Branching from existing topic - Calculating WAIC for models with multiple likelihood functions
I would like to consider the possibility of combining multiple log likelihoods for model comparison. Here is the example case that was mentioned by @OriolAbril in the previous thread - say we have TESS and Spitzer points or observations (say 30 and 13 respectively). Since the two instruments can have different systematics, jitter, etc., we would prefer to define separate log-likelihood functions for the different instruments. We would be combining them because the two instruments can have different sensitivities, characteristics, etc., which enable us to probe different regions of the astrophysical parameter space. For example their cadence, or on-sky sensitivity for data gathering might be different. In many cases where we have ground based observations, there can be more than 2 log-likelihood functions as well.
A few potential use cases -
-
Compare the log-likelihood and fit across different instruments, i.e. does the TESS data fit the astrophysical signal better than the Spitzer data? For a real case, this almost certainly would be the case with different instrumental wavelength coverage.
-
Combine the instrumental datasets to do model comparison across different runs… This relates to the previous thread where we can use log-likelihood value, WAIC, LOO, bayes factor, bayesian evidence (along with their uncertainties) to infer if one model is significantly preferred to another. It could be in terms of number of planets, circular vs eccentric orbits, or to compare other orbital parameters. This can naively be performed in terms of significance of the posteriors for additional components, however it would also be useful to approach this from a model comparison point of view.
The complication here is that the almost always the different datasets will have different sizes, and in terms of time series will rarely have any overlap, i.e. they will not be contemporaneous observations.