Pymc3 bayesian neural network for continuous regression

Hi,

I am stan user and got very excited with BNNs in pymc3. However, the training results are very poor. Can anybody tell if my model is wrong? The code is attached.cesun.py (4.5 KB)
I have copied twiecki/WhileMyMCMCGentlySamples example from git and replaced logistic regression (Bernoulli) with continuous (Normal) and added standard deviation parameter on output.

Thanks a lot.

How did the ELBO history looks like? Does it look converged?
It might be better to put your code in a notebook, then the result is visible and easier to see the fit etc.

ELBO looked converged. I am new to variational inference so don’t know what’s converged on not. I guess I should look for some stationary stuff but maybe there are better ways to determine convergence.

Jupyter is new for me but probably I need to learn it as well if it is a standard of communication.

My main concern is that I possibly did something wrong when converting logistic regression to continuous regression, adding biases, adding sd parameter.

Usually we just visually check whether the ELBO reach to a plateau. You can have a look at this discussion for an example: Poor Accuracy of ADVI for Linear Regression

Jupyter notebook is great! It’s like R notebook. I would say it is fairly easier to learn and you would love it.

If you are concern that you added biased while converting to continuous regression, you can set those parameters to a set value (ie, just set sd=.001).