A few quick thought:
- you should use a softmax instead of doing
T.repeatto to expand the weight to a right shape. - the choice of priors are a bit odd to me - is there any reason that you are using Uniform(0, 5) for betas?
- I think you want to model
forgetandalphato be in [0, 1] right? The current way you are doing might not give you parameter that satisfy such constraint. - maybe the
theano.scanpart could be rewrite into something doesnt need the scan?