TypeError: object of type 'MeanField' has no len() during pm.sample_posterior_predictive

Questions
1

Hi,
I am trying to do Thomas Wiecki’s [Bayesian Deep Learning Part II] (https://blog.quantopian.com/bayesian-deep-learning2/)

When I do the sample_posterior_predictive I get the TypeError: object of type ‘MeanField’ has no len() : error.

Is this a bug or am I doing something wrong.?

What I do is:

with neural_network:
inference=pm.ADVI()
trace=pm.fit(n=150, method=inference, score=True)

with neural_network:
ppc = pm.sample_posterior_predictive(trace, progressbar=False)

The code of the neural network:

def build_ann(init):

with pm.Model() as neural_network:

    l_in = lasagne.layers.InputLayer(shape=(None, 1, 28, 28),
                                 input_var=input_var)

    # Add a fully-connected layer of 800 units, using the linear rectifier, and
    # initializing weights with Glorot's scheme (which is the default anyway):
    n_hid1 = 800
    l_hid1 = lasagne.layers.DenseLayer(
        l_in, num_units=n_hid1,
        nonlinearity=lasagne.nonlinearities.tanh,
        b=init,
        W=init
    )

    n_hid2 = 800
    # Another 800-unit layer:
    l_hid2 = lasagne.layers.DenseLayer(
        l_hid1, num_units=n_hid2,
        nonlinearity=lasagne.nonlinearities.tanh,
        b=init,
        W=init
    )

    # Finally, we'll add the fully-connected output layer, of 10 softmax units:
    l_out = lasagne.layers.DenseLayer(
        l_hid2, num_units=10,
        nonlinearity=lasagne.nonlinearities.softmax,
        b=init,
        W=init
    )

    prediction = lasagne.layers.get_output(l_out)

    # 10 discrete output classes -> pymc3 categorical distribution
    out = pm.Categorical('out', prediction, observed=target_var)

return neural_network

class GaussWeights(object):
def init(self):
self.count = 0
def call(self, shape):
self.count += 1
return pm.Normal(‘w%d’ % self.count, mu=0, sd=.1,
testval=np.random.normal(size=shape).astype(np.float64),
shape=shape)

input_var = pm.Minibatch(X_train, batch_size=32)
target_var = pm.Minibatch(y_train, batch_size=32)

2

Object returns from pm.fit is not a MultiTrace like object, you should do something like:

mean_field = pm.fit(...)
trace = mean_field.sample(500)

For more see:
https://docs.pymc.io/notebooks/variational_api_quickstart.html

1 Like
3

Thanks
Could you also give me a hint on how to predict on a hold out set?
I don’t understand how to implement what is described in https://docs.pymc.io/notebooks/api_quickstart.html for advi.

I do use minibatches but want to use the sample_posterior_predict on the holdout data.

input_var = pm.Minibatch(X_train, batch_size=32)
target_var = pm.Minibatch(y_train, batch_size=32)

neural_network = build_ann(GaussWeights())

with neural_network:
inference=pm.ADVI()

 mean_field = pm.fit(n=500, method=inference, score=True)
 
 trace = mean_field.sample(500)

input_var = theano.shared(X_test[:50, …].astype(np.float64))
target_var = theano.shared(y_test[:50, …].astype(np.float64))

with neural_network:
ppc = pm.sample_posterior_predictive(trace)

But ppc[‘out’] has the shape 500,32 (seems to be from the minibatches)