Not able to obtain Predicted values in Bayesian neural network


I am trying to construct Bayesian neural network to solve my problem, with reference to the PyMC3 tutorial case “Variational Inference: Bayesian Neural Networks”. I am using Continuous dependent variable but not able to obtain the predicted values in Bayesian neural network.

Code used for construction of neural network :

def construct_nn(ann_input, ann_output):
    n_hidden = 5

    # Initialize random weights between each layer
    init_1 = np.random.randn(X.shape[1], n_hidden).astype(floatX)
    init_2 = np.random.randn(n_hidden, n_hidden).astype(floatX)
    init_out = np.random.randn(n_hidden).astype(floatX)

    with pm.Model() as neural_network:
        # Weights from input to hidden layer
        weights_in_1 = pm.Normal('w_in_1', 0, sd=1,
                                 shape=(X.shape[1], n_hidden),

        # Weights from 1st to 2nd layer
        weights_1_2 = pm.Normal('w_1_2', 0, sd=1,
                                shape=(n_hidden, n_hidden),

        # Weights from hidden layer to output
        weights_2_out = pm.Normal('w_2_out', 0, sd=1,

        # Build neural-network using tanh activation function
        act_1 = pm.math.tanh(,
        act_2 = pm.math.tanh(,
        act_out = pm.math.sigmoid(,
        out = pm.Normal('out',mu=act_out, observed=ann_output)
    return neural_network

ann_input = theano.shared(X_train)
ann_output = theano.shared(Y_train)
neural_network = construct_nn(ann_input, ann_output)

from pymc3.theanof import set_tt_rng, MRG_RandomStreams

#Fitting the model
with neural_network:
    inference = pm.ADVI()
    approx =, method=inference)

trace = approx.sample(draws=5000) 

After this step, i want to obtain the predicted values for a continuous variable.


Have a look at:

You can use:

preds = pm.sample_ppc(trace, model=model, samples=100) 
y_preds = preds['y']

to produce sample predictions on the data you used to train the model.

If you need to make predictions using other data as input, you can swap the data into ann_input since you use a shared Theano layer, and then run the sample_ppc