TypeError: object of type 'ADVI' has no len() when calling sample_posterior_predictive

Questions
1

Hi,

I am still working on the MNIST example from Thomas Wiecki.

When I try to do the predictions I get the error below.
I googled a bit and saw that there were bugs in this area. But as far as I can see they should be fixed.
Am I doing something wrong?

Regards Hans

TypeError Traceback (most recent call last)
in
1 with neural_network:
----> 2 ppc = pm.sample_posterior_predictive(inference, samples=500, progressbar=False)

/u01/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample_posterior_predictive(trace,

samples, model, vars, size, random_seed, progressbar)
1102 posterior predictive samples.
1103 “”"
-> 1104 len_trace = len(trace)
1105 try:
1106 nchain = trace.nchains

TypeError: object of type ‘ADVI’ has no len()

2

Can you share a bit more context? My guess is that you still have to call

trace = approx.sample(draws=5000)

before calling pm.sample_posterior_predictive.

For context, the ADVI object stores the fitted parameters of the variational family, and .sample will very quickly produce draws.

3

Sure,

the neural network:
def build_ann(init, input_var, target_var):

with pm.Model() as neural_network:

    l_in = lasagne.layers.InputLayer(shape=(None, 1, 28, 28),
                                 input_var=input_var)

    # Add a fully-connected layer of 800 units, using the linear rectifier, and
    # initializing weights with Glorot's scheme (which is the default anyway):
    n_hid1 = 800
    l_hid1 = lasagne.layers.DenseLayer(
        l_in, num_units=n_hid1,
        nonlinearity=lasagne.nonlinearities.tanh,
        b=init,
        W=init
    )

    n_hid2 = 800
    # Another 800-unit layer:
    l_hid2 = lasagne.layers.DenseLayer(
        l_hid1, num_units=n_hid2,
        nonlinearity=lasagne.nonlinearities.tanh,
        b=init,
        W=init
    )

    # Finally, we'll add the fully-connected output layer, of 10 softmax units:
    l_out = lasagne.layers.DenseLayer(
        l_hid2, num_units=10,
        nonlinearity=lasagne.nonlinearities.softmax,
        b=init,
        W=init
    )

    prediction = lasagne.layers.get_output(l_out)

    # 10 discrete output classes -> pymc3 categorical distribution
    out = pm.Categorical('out', prediction, observed=target_var)

return neural_network

#Creating the NN
neural_network = build_ann(GaussWeights(),minibatch_x, minibatch_y)

with neural_network:
inference = pm.ADVI()
approx = pm.fit(n=15000, method=inference, score=True)

0%| | 0/15000 [00:00<?, ?it/s]/u01/anaconda3/lib/python3.6/site-packages/theano/tensor/subtensor.py:2197: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
rval = inputs[0].getitem(inputs[1:])
Average Loss = 51,094: 100%|██████████| 15000/15000 [1:30:31<00:00, 2.64it/s]
Finished [100%]: Average Loss = 50,890

input_var.set_value(X_test)
target_var.set_value(y_test)

with neural_network:
ppc = pm.sample_posterior_predictive(inference, samples=500, progressbar=False)

TypeError Traceback (most recent call last)
in
1 with neural_network:
----> 2 ppc = pm.sample_posterior_predictive(inference, samples=500, progressbar=False)

/u01/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample_posterior_predictive(trace, samples, model, vars, size, random_seed, progressbar)
1102 posterior predictive samples.
1103 “”"
-> 1104 len_trace = len(trace)
1105 try:
1106 nchain = trace.nchains

TypeError: object of type ‘ADVI’ has no len()

4

Oof, this is strange action from a distance, but it looks like you need to:

trace = approx.sample(1000)  # or however many. doesn't need to be in Model context.

after calling pm.fit(...).

Then, after setting values on input_var and target_var, the proper call is

with neural network:
    ppc = pm.sample_posterior_predictive(trace, samples=500, progressbar=False)
5

Hi,
Thanks a lot.
I am still struggling with understanding pymc3.
Escpecially the correct sequence to make predictions after training a Bayesian NN.

What I actually want is get predictions of the MNIST.

So I create the minibatches:
input_var = pm.Minibatch(X_train, batch_size=32)
target_var = pm.Minibatch(y_train, batch_size=32)

Build the model:
neural_network = build_ann(GaussWeights()) # have removed input_var and target_var from the parameter list
Do the fitting.
with neural_network:
inference = pm.ADVI()
approx = pm.fit(n=15000, method=inference, score=True)

Do the sampling :
trace = approx.sample(draws=500) # is the number of draws related to the test set size??

Predict with X_test and compare the result with y_test.

ppc = pm.sample_posterior_predictive(trace, model=neural_network, progressbar=True)

Some how ppc[‘out’] has shape (500,32)

Should I set input_var and target_var to X_test and y_test after approx.sample…?

Regards Hans