I have the following Deep Learning model: X = theano.shared(X_train) Y = theano.shared(np.where(y_train==1)[1]) X_off = theano.shared(X_offset_train) h = [n_features, 10, 10, 1] inits = [] for i in range(len(h)-1): inits.append(np.random.randn(h[i], h[i+1])* np.sqrt(2/(h[i] + …

[image] sachinruk: I understand that VI is different in that we are optimising the mu/ sigma of the approximate posterior (q(w)), so maybe we had a large mu value? I see, yes you can track the mu/sigma value, see http://docs.pymc.io/notebooks/variational_api_quickstart.html#Tracking-parameter…

NaN occurred in optimization (error in DL with variational minibatches)

Questions

junpenglao March 20, 2018, 7:56am 2

Have you tried the methods to debug in this post?

Topic		Replies	Views
NaN occurred in optimization with ADVI Questions	5	4581	August 6, 2019
NaN occurred in optimization at first Iteration with ADVI Questions	3	751	February 18, 2020
How to use pymc3.fit() method Questions	16	3804	May 7, 2018
ADVI changes start values to NaN Questions	2	874	January 18, 2021
Pymc3 variational inference for multi-level logistic regression returning approximation equal to NaN Questions	2	609	April 12, 2021

NaN occurred in optimization (error in DL with variational minibatches)

Related topics