Stochastic node from a trace - or sharing params across models

Questions
1

Hi,

I’m trying to use the posterior of some parameters, \theta_1, from one model, \rm M_1, as stochastic variables in another model, \rm M_2, but not to update these \theta_1 parameters in the second model (which will be used to train some other parameters, \theta_2).

I found this thread: Prevent prior from updating?
which creates a ‘stochastic node’ in theano, but I’m wondering if this can be extended beyond univariate, parametric as shown in the example? My \theta_1 are a small array of params with some correlation between each other and not completely normally distributed.

Ideally, there would be some way to sample from some multi-variate kde of these variables in my second model (maybe borrowing from: Prior propagation by nonparametric copulas), but as a backup, I could try to see if a MvNormal parameterization describes the posteriors well. In either case, I need some way of sampling these as multi-variate, stochastic nodes in \rm M_2, which are not inferred params in \rm M_2.

Some pseudo-code to describe what I’m trying to do:

import pymc3 as pm

# Dataset 1
X = some data
Y = some data
# Dataset 2
W = some data
Z = some data

with pm.Model() as Model_1:
    # define priors
    theta_1 = pm.Normal('theta_1', mu=0, sigma=1, shape=6)
    err = pm.HalfNormal('err', sigma=1)

    # expectation function
    expect = f(theta_1, X)

    # likelihood function
    y_like = pm.Normal('y_like', mu=expect, sigma=err, observed=Y)

    trace = pm.sample(1000)

with pm.Model() as Model_2:
    # define priors
    theta_2 = pm.Normal('theta_1', mu=0, sigma=1, shape=2)
    err_2 = pm.HalfNormal('err_2', sigma=1)

    # expectation function
    expect_2 = g(trace['theta_1'], theta_2, W)

    # likelihood function
    z_like = pm.Normal('z_like', mu=expect_2, sigma=err_2, observed=Z)

    trace_2 = pm.sample(1000)

To head this question off, I don’t want to combine the two models for two reasons:

  1. The params \theta_1 are useful in quite a few other models, and it becomes unwieldy to combine many models with separate purposes.
  2. I believe f(\theta_1,X) in \rm M_1 to be a strong model, but g(trace[’\theta_1’], \theta_2, W), or variants thereof in other models may not be good models. I don’t want \theta_1 to be skewed by bias introduced by a poor choice of g().

Thanks in advance,
Daniel

2

I was trying to come up with a way to sample from another distribution using the following code but I ran into an error which I’m a little perplexed by. See below if it helps it all. The general idea is to make a step function which just makes a draw from a preexisting array.

import pymc3 as pm
import numpy as np

from pymc3.step_methods.arraystep import BlockedStep
external_vals = np.random.randn(10,2) * 5 + 2

class ExternalSample(BlockedStep):
    def __init__(self, external, var):
        self.vars = [var]
        self.name = var.name
        self.external = external

    def step(self, point):
        row_chosen = np.random.choice(self.external.shape[0])
        point[self.name] = self.external[row_chosen]       
        return point
    
with pm.Model() as model:
    external_pm = pm.Normal('external', shape=2)
    y = pm.Normal('y', mu=external_pm, shape=2)
    step = [ExternalSample(external_vals, external_pm)]
    trace = pm.sample(step=step)
3

Thanks - I’ll check this out. I’m also wondering what is happening under the hood when I set the observed flag of pm.Normal to True? Is this treating that variable as a stochastic node at that point?