How to predict on hold out set with variational api

Questions
1

Hi,

I am working through Thomas Wiecki’s Bayesian Deep Learning Part II: Bridging PyMC3 and Lasagne to build a Hierarchical Neural Network
Since pymc3 changed since then I need to modify the code slightly.

I want to predict on the hold out set.
But when I modify the input_var from :
input_var = pm.Minibatch(X_train, batch_size=32)
to:
input_var = theano.shared(X_test[:50, ...].astype(np.float64))
the shape of ppc[‘out’] is still (500, 32)

Am I doing something wrong?

I read the documentation but I failed to figure out how to do this right.

Regards Hans

input_var = pm.Minibatch(X_train, batch_size=32)
target_var = pm.Minibatch(y_train, batch_size=32)

neural_network = build_ann(GaussWeights())

with neural_network:
     inference=pm.ADVI()
     mean_field = pm.fit(n=500, method=inference, score=True)
     trace = mean_field.sample(500)

input_var  = theano.shared(X_test[:50, ...].astype(np.float64))
target_var = theano.shared(y_test[:50, ...].astype(np.float64))

**with neural_network:**
**    ppc = pm.sample_posterior_predictive(trace)**

**ppc['out'].shape **
**(500, 32)**


def build_ann(init):
    
    with pm.Model() as neural_network:
    
        l_in = lasagne.layers.InputLayer(shape=(None, 1, 28, 28),
                                     input_var=input_var)

        # Add a fully-connected layer of 800 units, using the linear rectifier, and
        # initializing weights with Glorot's scheme (which is the default anyway):
        n_hid1 = 800
        l_hid1 = lasagne.layers.DenseLayer(
            l_in, num_units=n_hid1,
            nonlinearity=lasagne.nonlinearities.tanh,
            b=init,
            W=init
        )

        n_hid2 = 800
        # Another 800-unit layer:
        l_hid2 = lasagne.layers.DenseLayer(
            l_hid1, num_units=n_hid2,
            nonlinearity=lasagne.nonlinearities.tanh,
            b=init,
            W=init
        )

        # Finally, we'll add the fully-connected output layer, of 10 softmax units:
        l_out = lasagne.layers.DenseLayer(
            l_hid2, num_units=10,
            nonlinearity=lasagne.nonlinearities.softmax,
            b=init,
            W=init
        )
    
        prediction = lasagne.layers.get_output(l_out)
    
        # 10 discrete output classes -> pymc3 categorical distribution
        out = pm.Categorical('out', prediction, observed=target_var)
    
    return neural_network
2

Did you try the solution here: https://github.com/pymc-devs/pymc3/issues/2190#issuecomment-311609342?

1 Like
3

Hi,
I had not seen it but tried it.
This works:

input_var = theano.shared(X_train[:5000, …].astype(np.float64))
target_var = theano.shared(y_train[:5000, …].astype(np.float64))

input_var_mb = pm.Minibatch(X_train, batch_size=32, dtype=np.float64)
target_var_mb = pm.Minibatch(y_train, batch_size=32, dtype=np.float64)

neural_network = build_ann(GaussWeights())

with neural_network:
inference=pm.ADVI()
mean_field = pm.fit(n=15000, method=inference, score=True, more_replacements={input_var: input_var_mb, target_var:target_var_mb})

trace = mean_field.sample(500)

with neural_network:
ppc = pm.sample_posterior_predictive(trace)

input_var.set_value(X_test[:5000, …].astype(np.float64))
target_var.set_value(y_test[:5000, …].astype(np.float64))

with neural_network:
ppc = pm.sample_posterior_predictive(trace)

Thanks

2 Likes
4

Hi,

Technically it works but the predictions are terrible.
The accuracy is about 10%.

Regards Hans