Predicting with Categorical

Questions
1

Hello,

I have 2 questions and would very grateful for some help.

  1. In the below model how do I change the p parameter in the Categorical for out of sample prediction? I have attempted with a shared variable but it doesn’t seem to do anything.

  2. Say my out of sample data had 4 data points instead of the 20 in the training. How can I get categorical to swallow the dimension change?

Many thanks!!

from theano import shared
import pymc3 as pm
import numpy as np

x_data = shared( np.random.rand(20,5,3) )
y_data = shared( 2 * np.argmax(x,axis=2) + (np.random.rand(20,5) -0.5) )

with pm.Model() as cmodel:
    c = pm.Categorical('cat',p=x_data,shape=(20,5))
    beta = pm.Exponential('beta',1)
    y = pm.Normal('y',mu=beta*c,sd=1,observed=y_data)
    t1 = pm.sample()

    #predict
    res1 = pm.sample_posterior_predictive(t1,100)
    
    # favour category 0
    c0 = np.zeros((20,5,3)) + 0.01
    c0[:,:,0] = 1
    x_data.set_value(c0)
    
    #predict again
    res2 = pm.sample_posterior_predictive(t1,100)
    
print( res1['y'].mean(0).mean() , res2['y'].mean(0).mean() )
2

Hi! This is not yet a direct answer to your question, but the API for out-of-sample predictions was simplified in a recent version. Maybe this will solve your problem?

3

Thanks for pointing me to this, it does seem a nice way to manage the data Unfortunately, I gave it a go but got the same results.

4

Was worth a try :wink:
Can you give more details about what you’re trying to model? I don’t really see how your likelihood relates to the data-generating process

5

Thanks for replying again, this is just a toy example. The answer to my original question is

  1. Delete the categorical variable with trace.remove_values('cat')
  2. Use a model factory like as demonstrated in How do we predict on new unseen groups in a hierarchical model in PyMC3? (this takes care of the shape problem)

But then it’s still slow which is known problem so the right thing to do is just ditch the Categorical altogether Marginalized Gaussian Mixture Model

1 Like
6

Yeah, sampling from a discrete distribution is usually impractical…
Glad you found a solution then – sorry I wasn’t more helpful :smile: