I have the following Deep Learning model: X = theano.shared(X_train) Y = theano.shared(np.where(y_train==1)[1]) X_off = theano.shared(X_offset_train) h = [n_features, 10, 10, 1] inits = [] for i in range(len(h)-1): inits.append(np.random.randn(h[i], h[i+1])* np.sqrt(2/(h[i] + …

[image] sachinruk: I understand that VI is different in that we are optimising the mu/ sigma of the approximate posterior (q(w)), so maybe we had a large mu value? I see, yes you can track the mu/sigma value, see http://docs.pymc.io/notebooks/variational_api_quickstart.html#Tracking-parameter…

NaN occurred in optimization (error in DL with variational minibatches)

Questions

junpenglao March 20, 2018, 7:56am 2

Have you tried the methods to debug in this post?

Topic		Replies	Views
FloatingPointError: NaN occurred in optimization in HMM Questions	5	1046	November 18, 2021
LDA example gives an Error Questions	3	471	August 13, 2018
How to use pymc3.fit() method Questions	16	3705	May 7, 2018
Minibatch Theano: shape mismatch: Questions	0	414	May 25, 2020
NaN occurred in optimization at first Iteration with ADVI Questions	3	730	February 18, 2020

NaN occurred in optimization (error in DL with variational minibatches)

Related topics