Why doesn't ADVI sample from my prior edges?

Questions v5
1

I have a basic distribution like this:

x_i = pm.Uniform("x", lower=-1.0, upper=0.0, shape=N, dims="x []")

When I sample using pm.fit() then pm.sample_approx(), the edges of the prior region, which are of course sharp, are not reached by my samples, which ends up biasing the posterior.

Does pymc or VI have known issues with prior edges?

Cross-posted on stackoverflow here pymc3 - In pymc, why isn't sampling from pm.Uniform giving a uniform distribution using VI .fit()? - Stack Overflow

Thanks!

2

Welcome!

Can you provide a description or visualization of what exactly you are seeing and how it deviates from what you expect?

1 Like
3

priors
priors599×789 112 KB

Like this!

4

Can you provide the code you used to generate the advi plot?

5

Isn’t this just a natural consequence of approximating a uniform distribution with a normal one (which is what ADVI does)?

6 
from matplotlib.pyplot import hist
from scipy.stats import uniform, norm

fit_mu, fit_sigma = norm.fit(uniform.rvs(size=10_000))
hist(norm(fit_mu, fit_sigma).rvs(10_000), bins=100, ec="k");

image
image552×413 4.97 KB

[Edit:] Hmm MLE includes the edges. The ADVI loss may penalize it more, would need to touble check

How does a 1D uniform look like?