Combining two models and contexts

Questions
1

Hi,

I have two different models trained on two different datasets.
I need to build a third model which combines parts of the two models There is no observed data/Likelihood, just some pm.Deterministic which are based on distributions of the two initial models.

Here is a stripped down example

    with pm.Model() as model1:
        mu1 = pm.Normal('mu1', mu=1, sd=1)
        like1 = pm.Normal('like1', mu=mu1, sd=1.0, observed=data1)
        model1_advi = pm.ADVI()
        model1_approx = model1_advi.fit()

    with pm.Model() as model2:
        mu2 = pm.Normal('mu2', mu=1, sd=1)
        like2 = pm.Normal('like2', mu=mu2, sd=1.0, observed=data2)
        model2_advi = pm.ADVI()
        model2_approx = model2_advi.fit()

    with pm.Model() as model3:
        interested_in = pm.Deterministic('interested_in', some_function(mu1, mu2))

I have some questions about this approach.

  1. How can I get the values of interested_in?
  2. If I fit model3, are the log likelihoods combined into a single one? Or are they treated independently? I would also need to give model3 as model parameter to model1 and model2.
  3. Is there a way to sample from the deterministic interested_in without fitting?

Thank you!

2

For question 2 and 3:
There is no way to sample from model 3, because there is no free parameters in the model and there are no log likelihood. Thus you cannot fit model 3, and parameters that appear in model 1 and 2 cannot be combined in model 3 (you will get disconnected error from theano if you are trying to use RVs that are defined in another model in a new model)

In general, if you have some function that computed on the MCMC samples, it is valid to do so if the RVs are from the same model. In another word, you should try to rewrite the model 1 and model 2 into one model and sample from it. Then you can get interested_in by computing some_function(mu1, mu2).
More boardly, if you are computing the expectation of some function \mathop{\mathbb{E}}f(\mu_1, \mu_2) you need to make sure the integration happens in the joint space of \mu_1 and \mu_2.