Returning posterior predictive probabilities for multiple categories

Questions
1

I’ve been searching for similar topics on my issue, could only find this.

I’m trying to sample the classification probabilities from the posterior for a set of test data, and using this I’m interested in measuring prediction uncertainty. So far I am only able to sample from the pm.Categorical output, however these class estimates have limited info that can be used to measure uncertainty.

For what it’s worth, here’s how I define my model:

def build_ann(init):
    with pm.Model() as model:

        network = lasagne.layers.InputLayer(shape=(None, 1, 28, 28),
                                        input_var=input_var)

        network = lasagne.layers.Conv2DLayer(  
            network, num_filters=32, filter_size=(5, 5),
            nonlinearity=lasagne.nonlinearities.tanh,
            W=init)

        network = lasagne.layers.MaxPool2DLayer(network, pool_size=(2, 2))    

        network = lasagne.layers.Conv2DLayer(
        network, num_filters=32, filter_size=(5, 5),
        nonlinearity=lasagne.nonlinearities.tanh,
        W=init)

        network = lasagne.layers.MaxPool2DLayer(network, pool_size=(2, 2))

        n_hid2 = 256

        network = lasagne.layers.DenseLayer(
        network, num_units=n_hid2,
        nonlinearity=lasagne.nonlinearities.tanh,
        b=init,
        W=init)

        network = lasagne.layers.DenseLayer(
        network, num_units=10,
        nonlinearity=lasagne.nonlinearities.softmax,
        b=init,
        W=init
        )
   
        prediction = lasagne.layers.get_output(network)
        
        out = pm.Categorical('out', 
                         prediction,
                         observed=target_var,total_size=int(y_train.shape[0]))

        return model

Any help on this would be greatly appreciated, as my current research depends on these results.

2

In your case, if you do posterior prediction using pm.sample_posterior_predictive, you can get the predicted observation, and you can use these posterior predictive samples to estimate the uncertainty (computing the variance, for example). Is it what you are looking for?

3

Apologies for the late reply. By using pm.sample_posterior_predictive I am only able to sample categorical outputs, over which we cannot compute values such as variance.

I did, however, manage to sample the probabilities from the model output using

sample_proba = approx.sample_node(model.out.distribution.p,
                                   size=n,
                                   more_replacements={input_var: x})
2 Likes