Hi every one.

I want to predict time series numbers through a neural network and pymc.my simple dataset consist of sequence numbers which I want to predict fourth number (i.e. 10,15,20->25).

I have modeled it with pymc3 linear regression and also raw theano (lasagna) neural network (without pymc3). in order to use pymc and lasagna, I followed this tutorial except for regression goal, not classification. The issue is model not learn at all and predict almost same numbers of all inputs.

Here is my model:

def build_ann(init):

l_in = lasagne.layers.InputLayer(shape=(None,3),

input_var=input_var)

# Add a fully-connected layer of 800 units, using the linear rectifier, and

# initializing weights with Glorot’s scheme (which is the default anyway):

n_hid1 = 4

l_hid1 = lasagne.layers.DenseLayer(

l_in, num_units=n_hid1,

nonlinearity=lasagne.nonlinearities.rectify,

W=init,

b=init

)

n_hid2 = 8

l_hid2 = lasagne.layers.DenseLayer(

l_hid1, num_units=n_hid2,

nonlinearity=lasagne.nonlinearities.rectify,

W=init,

b=init

)

l_out = lasagne.layers.DenseLayer(

l_hid2, num_units=1,

W=init,

b=init

)

prediction = lasagne.layers.get_output(l_out)

# pdb.set_trace()

sigma = pm.HalfNormal(“sigma”, sigma=1)

out = pm.Normal(‘out’,

mu=prediction,

sigma=sigma,

observed=target_var)

return out

and initialize weight with

class GaussWeights(object):

definit(self):

self.count = 0

defcall(self, shape):

self.count += 1

return pm.Normal(‘w%d’ % self.count, mu=0, sd=.1,

testval=np.random.normal(size=shape).astype(np.float64),

shape=shape)

and

with pm.Model() as neural_network:

likelihood = build_ann(GaussWeights())

with neural_network:

infrence=pm.ADVI()

mean_field=pm.fit(100000,method=infrence)

trace=mean_field.sample(5000)