Bayesian Neural Net with Beta distribution

Questions
1

Hello there,

I’ve got a question regarding Bayesian Neural Nets which are described here

Say I want the distribution for the weights to be a beta distribution instead of a normal distribution for whatever reason.

Then I can do variational inference and sample from the approximation, but my posterior predictive check fails with an error message:

raise MissingInputError(“Undeclared input”, variable=variable)
theano.gof.fg.MissingInputError: Undeclared input

The Error can be traced back to the draw_values function in /distributions/distribution.py, where the dict “named_nodes” has extra values that aren’t there for normal distributions like ‘weights2_log__’ or which are named differently. I think the problem is, that those values can’t be found in the “points” dict at this moment.

My question is therefore: Can I somehow circumvent this problem respectively is there no solution because of the different nature of the distributions? If so, why are they so different?

Thank you in advance for your help!

2

It is possible that this is a bug in how draw_values handles transformed distributions. We made some changes to that pretty recently, and this might have introduced a new issue. Could you post some code that fails in this way, plus the version of theano and pymc3 you are using? That would make debugging much easier.

3

Yeah there is no random method for TransformedDistribution. We should probability add that.

4

Thanks for your replies!

Unfortunately I cannot post any code at the moment for debugging until monday, but the error should be reproducable by using the linked example and replacing the Normal distributions with Beta distributions.
I’m using the latest Theano and Pymc3 Version, since I recently upgraded the packages.

5

Oh I see, the reason is that the sample from the approximation does not include the transformed variable, you should be able to make it work by replacing the

trace = approx.sample(draws=5000)

with

trace = approx.sample(draws=5000, include_transformed=True)
2 Likes
6

That seems very promising to me, since the transformed variables aren’t present in the points dict.
I will give it a try as soon as I can and tell you about the results.

Thank you very much for your help!

7

That’s exactly what I needed.

Thanks again!