# Does sample\_posterior\_predictive() ignore arrays in the trace when used as model factory?

**URL:** <https://discourse.pymc.io/t/does-sample-posterior-predictive-ignore-arrays-in-the-trace-when-used-as-model-factory/6307>\
**Category:** Questions\
**Created:** [November 25, 2020, 3:35pm UTC](https://discourse.pymc.io/t/does-sample-posterior-predictive-ignore-arrays-in-the-trace-when-used-as-model-factory/6307 "2020-11-25T15:35:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![bridgeland](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/bridgeland/32/2773_2.png) [@bridgeland](https://discourse.pymc.io/u/bridgeland)\
**Post date:** [November 25, 2020, 3:35pm UTC](https://discourse.pymc.io/t/does-sample-posterior-predictive-ignore-arrays-in-the-trace-when-used-as-model-factory/6307/1 "2020-11-25T15:35:14Z")

</div>

@lucianopaz shows [how to use a model factory](https://discourse.pymc.io/t/how-do-we-predict-on-new-unseen-groups-in-a-hierarchical-model-in-pymc3/2571/2) to sample in a second model using values copied from the trace of a first model. (Thanks Luciano, for pioneering that valuable technique!)

When employing a model factory I am seeing a strange difference in the effect of copied arrays versus copied scalars. Considering this simple example:

```auto
import pymc3 as pm

def model_factory():
    """Create a model with a scalar normal and an array normal"""
    with pm.Model() as model:
        foo = pm.Normal('foo', 0, 1)
        foo_array = pm.Normal('foo_array', 0, 1, shape=10)
        pm.Deterministic('bar', foo)
        pm.Deterministic('bar_array', foo_array)
    return model

# Create a model and sample it
with model_factory() as antecedent:
    ant_trace = pm.sample(draws=600, chains=2, tune=500)

# Create a second model, copy the values for foo and foo_array
# to the model, and then sample posterior
with model_factory() as consequent:
    ant_trace_df = pm.trace_to_dataframe(ant_trace, varnames=['foo', 'foo_array'])
    conseq_trace = pm.sample_posterior_predictive(
        trace=ant_trace_df.to_dict('records'),
        var_names=['bar', 'bar_array'],
        samples=len(ant_trace_df))

```

This example creates two identical models, **antecedent** and **consequent**. **antecedent** is sampled, and the values for **foo** and **foo\_array** are copied to **consequent**. Then **consequent** is sampled.

Take a look at one sample (e.g. 42) in the traces of **antecedent** and **consequent**. Note that we cannot look at the value for **foo** in **consequent** , as it does not exist in the trace. But we can look at **bar** , a deterministic that takes the value of **foo**. The values for **bar** are identical in **antecedent** and **consequent** , as expected:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/9/90b5e9b894e788169e8347dbd8b0f6c38325d2a0.png)

**foo\_array** is just like **foo** , except it is an array instead of a scalar. And **bar\_array** is just like **bar** , except it is also an array instead of a scalar. But the values for **bar\_array** are completely different between **antecedent** and **consequent** :

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/7/786c7bf31d3c7f4e3c66396a4953f7ee6f129a08.png)

**sample\_posterior\_predictive** seems to be ignoring the values of **foo\_array** copied in from **antecedent** , and instead resampling the normal distribution.

Is this expected behavior? Is this a bug in **sample\_posterior\_predictive()**? (**fast\_sample\_posterior\_predictive()** exhibits the same behavior, albeit much faster.)

PyMC3 version 3.9.3

---

<div class="post-metadata">

**Author:** ![bridgeland](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/bridgeland/32/2773_2.png) [@bridgeland](https://discourse.pymc.io/u/bridgeland)\
**Post date:** [November 29, 2020, 1:36pm UTC](https://discourse.pymc.io/t/does-sample-posterior-predictive-ignore-arrays-in-the-trace-when-used-as-model-factory/6307/2 "2020-11-29T13:36:50Z")

</div>

[Reported as bug](https://github.com/pymc-devs/pymc3/issues/4272).

---

<div class="post-metadata">

**Author:** ![bridgeland](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/bridgeland/32/2773_2.png) [@bridgeland](https://discourse.pymc.io/u/bridgeland)\
**Post date:** [December 7, 2020, 12:04pm UTC](https://discourse.pymc.io/t/does-sample-posterior-predictive-ignore-arrays-in-the-trace-when-used-as-model-factory/6307/3 "2020-12-07T12:04:15Z")

</div>

[Elsewhere](https://github.com/pymc-devs/pymc3/issues/4272) @lucianopaz figured out the problem:

> The problem is caused by `pm.trace_to_dataframe(ant_trace, varnames=['foo', 'foo_array'])`. This creates a pandas dataframe where the entries for **foo\_array** are split across columns named **foo\_array\_0** , **foo\_array\_1** , etc. These are then passed on to **sample\_posterior\_predictive** but there is no way for the sampler to understand that **foo\_array\_0** , **foo\_array\_1** , and the rest should be stacked together into a single **foo\_array** value.
> 
> There are two ways to address this problem:

> 1. Don’t convert to a DataFrame and drop the values from the MultiTrace directly:

```python
with model_factory() as consequent:
    ant_trace.remove_values("bar")
    ant_trace.remove_values("bar_array")
    conseq_trace = pm.sample_posterior_predictive(
        trace=ant_trace,
        var_names=['bar', 'bar_array'],
    )

```

> 1. Use **arviz.InferenceData** that stores the posterior samples in an **xarray.Dataset** , that can understand the multidimensional **foo\_array** and **bar\_array** without problems.

```python
with model_factory() as antecedent:
    ant_trace = pm.sample(draws=600, chains=2, tune=500, return_inferencedata=True)

# Create a second model, copy the values for foo and foo_array
# from the first model to the second to the model. Then sample posterior
with model_factory() as consequent:
    ant_trace.posterior = ant_trace.posterior.drop_vars(["bar", "bar_array"])
    conseq_trace = pm.sample_posterior_predictive(
        trace=ant_trace,
        var_names=['bar', 'bar_array'],
    )

```