# Inf Average loss ADVI in Correlated Topic Model

Hi,

While implementing Correlated Topic Model with ADVI I get inf average loss. What does it mean? Now I’m unsure whether I’m implementing it in a correct way or not.

Here is my code:

``````import numpy as np
import pymc3 as pm, theano.tensor as t
import matplotlib.pyplot as plt
K = 4
V = 4 # number of words
D = 10 # number of documents

data = np.random.randint(V,size=(D,4))

alpha = np.ones((1, K))
beta = np.ones((1, V))
model = pm.Model()

mu = np.ones((1,K),dtype = np.float64)
cov = np.random.random_sample(size=(K,K))

Wd = [len(doc) for doc in data]
(D, W) = data.shape

def log_lda(theta,phi):
def ll_lda(value):
dixs, vixs = value.nonzero()
vfreqs = value[dixs, vixs]
ll =vfreqs* pm.math.logsumexp(t.log(theta[dixs]) + t.log(phi.T[vixs]), axis = 1).ravel()
return t.sum(ll)
return ll_lda

with model:
eta = pm.MvNormal('eta',mu = mu,cov = cov, shape = (D,K))
theta = t.nnet.softmax(eta)
phi = pm.Dirichlet("phi", a=beta, shape=(K, V))
doc = pm.DensityDist('doc', log_lda(theta,phi), observed=data)
with model:
approx = pm.fit(n=10000,method= inference,callbacks=[pm.callbacks.CheckParametersConvergence(diff='absolute')])

#inference
tr1 = approx.sample(draws=1000)
pm.plots.traceplot(tr1);
pm.plot_posterior(tr1, color='LightSeaGreen');

plt.plot(approx.hist)
``````

Can someone please guide me with this?
Help much appreciated.
Thanks

Average Loss = inf: 100%|██████████| 10000/10000 [00:06<00:00, 1497.31it/s]
Finished [100%]: Average Loss = nan

What is `plt.plot(approx.hist)` looks like?

It gives me nothing. Its blank.

Yeah there is definitely some problems of your model. You should double check the likelihood implementation, print the testpoint and `model.check_test_point`.

It gives me

``````eta -inf
phi_stickbreaking__ -15.01367190100603
doc -72.08730677823429
``````

For eta :

Using this I have used
`eta = pm.MvNormal('eta',mu = mu,cov = cov, shape = (D,K))`

Is there anything wrong with this?

This is not right - the reason is that your cov is not a positive definite matrix.

In the documentation provided, its not mentioned about the positive definiteness of the covariance matrix?

I guess the document is not explicit enough on that but the logp of MvNormal (or MvStudentT) only defined on covariance matrix that are positive definiteness.

Okay! Thanks.
Also do you know any implementation of Stochastic Mixed membership Block Models in pymc3. I want to implement it using ADVI.