Multilevel model where prior samples have unit standard deviation

Questions v5
1

I’m trying the set up a multilevel model so that prior samples from a child variable have a standard deviation of 1 (taking into account the dispersion of the parent variable). For example, in the below:

with pm.Model() as my_model:
    parent = pm.Normal('parent', mu=0.0, sigma=shared_sd)
    child = pm.Normal('child', mu=parent, sigma=shared_sd)

I want to set shared_sd such that the prior samples from child have an SD of 1. I couldn’t figure out a simple way to calculate shared_sd in closed form, so I tried to use pymc to estimate it for me as follows:

with pm.Model() as my_model:
    shared_sd = pm.HalfNormal('shared_sd', sigma=1.0)
    parent = pm.Normal('parent', mu=0, sigma=shared_sd)
    child = pm.Normal(
        'child', mu=parent, sigma=shared_sd,
        observed=np.random.normal(0, 1, 10000)
    )

    idata = pm.sample()

Sampling looks fine but I’m getting unexpected posteriors:

posteriors
posteriors808×303 15.1 KB

Shouldn’t the posterior variance of parent be dictated by shared_sd here?

1 Like
2

Welcome!

What exactly were you expecting that you’re not seeing?

Your model represents a bit of a paradox (not an actual paradox, but a bit awkward). In the limit of large data, one might hope that the SD of child should approach 1 (the true SD) and the SD of parent should approach 0 (because the true mean should be estimated with infinite precision). But each of these SDs is tied to a single parameter (shared_sd), which forces a compromise that is influenced by the amount of observed data.

1 Like
3

No, shared_sd defines the standard deviation of the prior. Depending on the data, the posterior will have different mean and/or standard deviation or even be a completely different distribution. The posterior on shared_sd is actually an example of the posterior being a different distribution: the prior is a half normal and the posterior is close to a normal.