PyMC3 with Keras

[ericmjl]

@sachinruk

x_in = Input(shape=(X_train.shape[1:]))
h = Dense(10, use_bias=False)(x_in)
out = Activation('softmax')(h)
model = Model(x_in, out)

Here’s an implementation using PyMC3 + Theano.

import theano.tensor as tt

with pm.Model() as nn_model:
    w1 = pm.Normal('w1', 0, 1, shape=(10,))
    h = tt.dot(x_in, w1)
    h = tt.nnet.softmax(h)
    like = pm.Normal('likelihood', mu=h, observed=data)

The key to understanding neural networks is that it is basically linear (affine) transforms of data + nonlinearity functions applied. Once you get this, Bayesian neural networks are nothing more than placing priors on those weights!