Continuous Bernoulli distribution

Questions
1

Currently, I’m working on a model which involves Bernoulli distributions. As I’m utilizing ADVI, it is not possible to utilize discrete/binary RVs. Therefore, I was trying to implement the continuous Bernoulli distribution: https://arxiv.org/abs/1907.06845 . There is also a wikipedia article based on the paper: https://en.wikipedia.org/wiki/Continuous_Bernoulli_distribution

However, I have some difficulties in implementing the distribution by using pm.DensityDist. My formulation is inspired by other posts here in the forum, still I get a TypeError. Below is a simplified model where I get the error. Is there anything I miss here?

 import numpy as np
 import pymc3 as pm
 from theano import tensor as T
 from theano.ifelse import ifelse
 
 def logp_cont_bernoulli(x, lam_psi):
     C = ifelse(T.eq(lam_psi, 1 / 2), np.float64(2), (2 * T.arctanh(1 - 2 * lam_psi)) / (1 - 2 * lam_psi))
 
     return T.log(C * lam_psi ^ x * (1 - lam_psi) ^ (1 - x))
 
 obs_data = np.array([1,0,0,0,0,0,1,1]).astype('int16')
 
 with pm.Model() as model:
     # prior
     lam_psi = pm.Bound(pm.Normal, lower=0.0, upper=1.0)('lam_psi', mu=0.3, sd=0.05, testval=0.3)
     # likelihood
     like = pm.DensityDist('like', logp_cont_bernoulli,
                               observed=dict(x=obs_data, lam_psi=lam_psi))
     # sampling
     trace = pm.sample(2000)
2

Could you explain a bit more of your use case? If the Bernoulli is observed, ADVI works as well (the continuous requirement is for the latent variables).
If Bernoulli is latent, then usually we will rewrite the model as a mixture model (i.e., marginalization trick)

3

Hey @junpenglao, thanks for your reply! I didn’t know that ADVI works with observed Bernoulli.

In fact, in my originary model, the Bernoulli is latent. The simplified exemplary model above was my first try to implement the continuous Bernoulli, even though it is observed in this case. So it seems, my example is not perfectly fitted to underline my question…

So I would be interested in your approach of using a mixture model for that purpose. Could you tell me a bit more about that or refer me to some sources?

4

There are some guides here: Frequently Asked Questions

1 Like