# Unable to make OOS predictions using a simple model

**URL:** <https://discourse.pymc.io/t/unable-to-make-oos-predictions-using-a-simple-model/11566>\
**Category:** v5\
**Created:** [March 8, 2023, 7:29pm UTC](https://discourse.pymc.io/t/unable-to-make-oos-predictions-using-a-simple-model/11566 "2023-03-08T19:29:10Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dekermanjian](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@Dekermanjian](https://discourse.pymc.io/u/Dekermanjian)\
**Post date:** [March 8, 2023, 7:29pm UTC](https://discourse.pymc.io/t/unable-to-make-oos-predictions-using-a-simple-model/11566/1 "2023-03-08T19:29:10Z")

</div>

Hello, everyone. I am having an issue with getting OOS predictions from my pymc model. If I use new data it throws a value error, and if I try only setting 1 observation as the data it works but returns predictions with a shape equal to the training data.

Here is my model:

> conversions = ag\_d[‘conversions’].to\_numpy()
> 
> conversions\_sq = np.square(conversions)
> 
> scaler = StandardScaler()
> 
> conversions\_sq\_scaled = scaler.fit\_transform(conversions\_sq.reshape(-1,1)).flatten()
> 
> conversions = conversions[:, None]
> 
> admits = ag\_d[‘admits’].to\_numpy()
> 
> with pm.Model() as linear:  
> X = pm.MutableData(“X”, conversions\_sq\_scaled)  
> alpha = pm.Exponential(“alpha”, 1/10)  
> b0 = pm.HalfNormal(‘intercept’, sigma=4)  
> b1 = pm.HalfNormal(‘beta’, sigma=2)  
> mu = pm.Deterministic(‘mu’, var=pm.math.exp(b0 + b1\*X))  
> l = pm.NegativeBinomial(“l”, mu=mu, alpha=alpha, observed=admits)
> 
> with linear:  
> idata\_linear = pm.sample(tune=2000, draws=4000, target\_accept=0.99)  
> with linear:  
> ppc\_linear = pm.sample\_posterior\_predictive(idata\_linear)  
> with linear:  
> pm.set\_data(new\_data={‘X’: new\_conversions\_sq\_scaled})  
> ppc\_oos = pm.sample\_posterior\_predictive(idata\_linear)

And the error I get:

> ValueError Traceback (most recent call last) File [c:\Users\j.dekermanjian\Anaconda3\envs\m\_geo\lib\site-packages\pytensor\compile\function\types.py:970](file:///C:/Users/j.dekermanjian/Anaconda3/envs/m\_geo/lib/site-packages/pytensor/compile/function/types.py:970), in Function. **call** (self, \*args, \*\*kwargs) 968 try: 969 outputs = ( → 970 self.vm() 971 if output\_subset is None 972 else self.vm(output\_subset=output\_subset) 973 ) 974 except Exception: File [c:\Users\j.dekermanjian\Anaconda3\envs\m\_geo\lib\site-packages\pytensor\graph\op.py:543](file:///C:/Users/j.dekermanjian/Anaconda3/envs/m\_geo/lib/site-packages/pytensor/graph/op.py:543), in Op.make\_py\_thunk…rval(p, i, o, n, params) 539 @is\_thunk\_type 540 def rval( 541 p=p, i=node\_input\_storage, o=node\_output\_storage, n=node, params=None 542 ): → 543 r = p(n, [x[0] for x in i], o) 544 for o in node.outputs: File [c:\Users\j.dekermanjian\Anaconda3\envs\m\_geo\lib\site-packages\pytensor\tensor\random\op.py:378](file:///C:/Users/j.dekermanjian/Anaconda3/envs/m\_geo/lib/site-packages/pytensor/tensor/random/op.py:378), in RandomVariable.perform(self, node, inputs, outputs) 376 rng\_var\_out[0] = rng → 378 smpl\_val = self.rng\_fn(rng, \*(args + [size])) 380 if ( 381 not isinstance(smpl\_val, np.ndarray) 382 or str(smpl\_val.dtype) != out\_var.type.dtype
> 
> …
> 
> Inputs values: [Generator(PCG64) at 0x272252051C0, array([72], dtype=int64), array(4, dtype=int64), array(13.6946887), ‘not shown’] Outputs clients: [[‘output’], [‘output’]] HINT: Re-running with most PyTensor optimizations disabled could provide a back-trace showing when this node was created. This can be done by setting the PyTensor flag ‘optimizer=fast\_compile’. If that does not work, PyTensor optimizations can be disabled with ‘optimizer=None’. HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.

I also try doing:

> with linear:  
> pm.set\_data(new\_data={‘X’: np.array([1.0])})  
> ppc\_oos = pm.sample\_posterior\_predictive(idata\_linear)

This runs but returns predictions with the same length as the training data. So I don’t think the model is using the new data.

Any advice would be appreciated. Also how do you put tabs in the code blocks?

EDIT:  
Actually, it looks like when I run it with only 1 sample it is broadcasting the value to the original training data size. This must be an issue with the shape/dims.

EDIT2:  
I changed my model to include the dimensions, but I am still getting the same error.  
Here is what I changed:

> with pm.Model() as linear:  
> linear.add\_coord(name=‘dims’, values=np.arange(0, len(conversions\_sq\_scaled)), mutable=True)  
> X = pm.MutableData(“X”, conversions\_sq\_scaled)  
> alpha = pm.Exponential(“alpha”, 1/10)  
> b0 = pm.HalfNormal(‘intercept’, sigma=4)  
> b1 = pm.HalfNormal(‘beta’, sigma=2)  
> mu = pm.Deterministic(‘mu’, var=pm.math.exp(b0 + b1\*X))  
> l = pm.NegativeBinomial(“l”, mu=mu, alpha=alpha, observed=admits, dims = ‘dims’)
> 
> with linear:  
> pm.set\_data(new\_data={‘X’: new\_conversions\_sq\_scaled}, coords={‘dims’: np.arange(0, len(new\_conversions\_sq\_scaled))})  
> ppc\_oos = pm.sample\_posterior\_predictive(idata\_linear)

EDIT3:  
Okay, I was able to get it to work.  
here is what I needed to change:

> with pm.Model() as linear:  
> X = pm.MutableData(“X”, conversions\_sq\_scaled)  
> alpha = pm.Exponential(“alpha”, 1/10)  
> b0 = pm.HalfNormal(‘intercept’, sigma=4)  
> b1 = pm.HalfNormal(‘beta’, sigma=2)  
> mu = pm.Deterministic(‘mu’, var=pm.math.exp(b0 + b1\*X))  
> l = pm.NegativeBinomial(“l”, mu=mu, alpha=alpha, observed=admits, shape=X.shape)  
> with linear:  
> pm.set\_data(new\_data={‘X’: new\_conversions\_sq\_scaled})  
> ppc\_oos = pm.sample\_posterior\_predictive(idata\_linear)

Can someone explain to me why the shape worked but the dims did not?