EDIT: Sorry – false alarm – I had incorrectly understood my posterior dependency structure. There is no question here and all is well! This is a “selection prior” model not a “dropout” specification…
I’m playing around with PyMC in a slightly strange way, but I’m surprised by the behavior I’m seeing; namely, it does not appear that missing values are imputed upon every posterior iteration. The final plot shows the binary status of 10 missing Bernoulli variables as they are imputed over 400 posterior iterations, and they are much more sticky than is possible when flipping a coin.
How are missing values imputed in PyMC? What’s the frequency/cadence of the imputation?
n = 100
x = np.random.normal(loc=10, size=n)
width,depth=10,1
d = np.zeros((width,depth)) # if we mask this it will be automatically imputed
d = np.ma.masked_array(d, mask = d==0) # via sampling it from its distribution
with pm.Model() as dropout_network:
dropout_rate = 0.5
dropout_layers = pm.Bernoulli("dropout", p=dropout_rate, shape=d.shape, observed=d)
mu = pm.Normal("prior", shape=d.shape)
pm.Normal("likelihood", mu=mu.T.dot(dropout_layers), sigma=1, observed=x)
step = pm.Slice([mu])
with dropout_network:
trace = pm.sample(200, step=step, tune=200, chains=2)
