I tried to infer following logistic regression model using ADVI,

```
def invlogit(self, x):
return tt.exp(x)/(1+ tt.exp(x))
with pm.Model() as self.logistic_model:
alpha = pm.Normal('alpha', mu=0, sd=20)
beta = pm.Normal('beta', mu=0, sd=20, shape=X.shape[1])
mu = alpha + tt.dot(X, beta)
p = pm.Deterministic('p', invlogit(mu))
y = pm.Bernoulli('y', p=p, observed=y)
apprx = pm.fit(1000)
```

Then I got the following error,

```
Average Loss = 1.0766e+07: 0%| | 0/10000 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/nadheesh/PycharmProjects/mcmc_vs_variational/mcmc/models.py", line 319, in <module>
lr.fit(X,y)
File "/home/nadheesh/PycharmProjects/mcmc_vs_variational/mcmc/models.py", line 193, in fit
apprx = pm.fit(10000, obj_optimizer=pm.adam(), obj_n_mc=20)
File "/home/nadheesh/anaconda3/envs/dev/lib/python3.5/site-packages/pymc3/variational/inference.py", line 756, in fit
return inference.fit(n, **kwargs)
File "/home/nadheesh/anaconda3/envs/dev/lib/python3.5/site-packages/pymc3/variational/inference.py", line 135, in fit
state = self._iterate_with_loss(0, n, step_func, progress, callbacks)
File "/home/nadheesh/anaconda3/envs/dev/lib/python3.5/site-packages/pymc3/variational/inference.py", line 181, in _iterate_with_loss
raise FloatingPointError('NaN occurred in optimization.')
FloatingPointError: NaN occurred in optimization.
```

I try to investigate this a little bit and found that this is cause by the NaN return by the step_function() when calculating the error. Moreover, If I change the Bernoulli distribution to a Normal distribution then model can be trained without any error. However, I can’t understand why this error is observed when using the Bernoulli likelihood.

I appreciate if someone can help me to resolve this issue.